quest quality management tool for urban energy efficient ... · 5 comparison of the approach of...

INTELLIGENT ENERGY EUROPE

Contract Number S12.589406

QUEST

Quality management tool for Urban Energy efficient Sustainable Transport

STATE OF THE ART OF URBAN MOBILITY ASSESSMENT

Deliverable no. D3.1

Dissemination level Public

Work Package 3, State of the art of urban mobility assessment

Author(s) Dr Philip Barham, Samantha Jones and Maja van der Voet

Co-author(s) Leisa Stephenson, Vicky Edge, Jürgen Perschon, Simon Field, Elisabeth Undén, Michael Koucky, Radomira Jordova, Dirk Engels, Natalie Craeghs and Eliene Van Aken

Status (F: final, D: draft) F1.5 – 03/01/2012

File Name QUEST_WP3_SoA of urban mobility assessment_FINAL.doc

Project Start Date and Duration 10 May 2011, 30 months

State of the art of urban mobility assessment Public

03/01/2012 2 TTR

Document history

Version Date Description/Changes

1 31.08.11 Initial draft containing introduction, methodology and most appendices

1.1 23.09.11 Second draft with most remaining sections added

1.2 28.09.11 Third draft for review

1.3 14.10.11 Fourth draft with revisions to sections 5 and 7

1.4 18.10.11 Final draft for confirmation

1.5 03.01.12 Final version for publication

Document flow

Sent Date

From S Jones to P Barham 31.08.11

From S Jones to P Barham 23.09.11

From P Barham to S Jones 27.09.11

From S Jones to WP3 staff and Project Management Team for review 28.09.11

From M van der Voet to P Barham/S Jones 14.10.11

From S Jones to WP3 staff and Project Management Team for confirmation 18.10.11

From M van der Voet to EACI 26.10.11

From EACI to M van der Voet 30.11.11

From P Barham to S Jones 16.12.11

From S Jones to WP3 staff and I Cré (final version) 03.01.12

Approval

By Approval date

Project Management Team 21.10.11

The sole responsibility for the content of this report lies with the authors. It does not necessarily reflect the opinion of the European Union.Neither the EACI nor the European Commission are responsible for any use that may be made of the information contained therein.


03/01/2012 3 TTR

TABLE OF CONTENTS

1 Introduction 13

1.1 A note on terminology 13

2 Methodology 14

2.1 Task 3.1: State of the art of goals and strategies at a European level 14

2.2 Task 3.2: State of the art of approaches to assessing sustainable urban mobility plans 14

2.3 Task 3.3: Review of projects using the concept of TQM to develop an audit and self-certification tool 15

2.4 General comment 16

3 Review of European legislation/policies relevant to SUMPs 17

3.1 Summary of findings from review of European legislation and policies 17

3.2 EC Action Plan on Urban Mobility (2009 - 2013) 18

3.3 EC White Paper on the Future of EU Transport Policy, Roadmap to a Single Transport Area – Towards a Competitive and Resource-Efficient Transport System (2011 – 2050) 21

3.4 Other important legislation and policy documents 22

3.4.1 EC Communication: A Resource-efficient Europe – Flagship Initiative under the Europe 2020 Strategy (2010 – 2020) 22

3.4.2 EC Thematic Strategy on the Urban Environment (2006 onwards) 23 3.4.3 Mainstreaming Sustainable Development into EU Policies – 2009 Review of the EU Strategy for

Sustainable Development (2009 onwards) 24 3.4.4 EC White Paper: Adapting to Climate Change – Towards a European Framework for Action (2009 -

2012) 24 3.4.5 Renewed EU Strategy for Sustainable Development (2006 onwards) 24 3.4.6 Energy Efficiency Plan (2011) 24 3.4.7 Ambient Air Quality Directive (2008 - On-going) 25 3.4.8 Energy End-use Efficiency and Energy Services Directive (2006 - On-going) 25 3.4.9 EC Directive on the Framework for the Deployment of Intelligent Transport Systems (ITS) in the field

of Road Transport and for Interfaces with Other Modes of Transport (2010 - 2017) 25 3.4.10 EC Action Plan for the Deployment of ITS in Europe (2008 - 2014) 25 3.4.11 Promotion of Clean and Energy Efficient Road Transport Vehicles (2009 – On-going) 25 3.4.12 The Greening Transport Strategy Package (2008 – On-going) 25

4 Review of approaches to evaluation 27

4.1 Summary of findings on approaches to evaluation 27

4.2 Findings of previous state of the art reviews and reports by the ELTIS+ project and DG Environment 28

4.2.1 State of the art review and guidelines from the ELTIS+ project 28 4.2.2 Report of the Expert Working Group on Sustainable Urban Transport Plans (December 2004) 32 4.2.3 Report: Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and

Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions (October 2005) 33

4.3 Approaches to evaluation: Current European projects and initiatives 33

4.3.1 PILOT (Planning Integrated Local Transport) 33 4.3.2 The CIVITAS evaluation programme 35 4.3.3 ARCHIMEDES (Achieving Real Change with Innovative Transport Measures Demonstrating Energy

Savings) 36 4.3.4 EPOMM PLUS (European Platform on Mobility Management PLUS), and the Max Project suite of

tools 37 4.3.5 GUIDEMAPS (Gaining Understanding of Improved Decision Making And Participation Strategies) 39 4.3.6 BUSTRIP (Baltic Urban Sustainable Transport Implementation and Planning) 40 4.3.7 DISTILLATE (Design and Implementation Support Tools for Integrated Local Land Use, Transport

and the Environment) 42 4.3.8 OPTIMUM II (Optimal Planning Through Implementation of Mobility Management) 44


03/01/2012 4 TTR

4.3.9 MOBILIS (Mobility Initiatives for Local Sustainability) 44 4.3.10 MOVE project 45 4.3.11 CATCH (Clean Accessible Transport for Community Health) 46 4.3.12 PROSPECTS (Procedures for Recommending Optimal Sustainable Planning of European City

Transport Systems) 47 4.3.13 CO2NeuTrAlp (CO2-Neutral Transport for the Alpine space) 48

4.4 National reports on the evaluation of SUMP and similar planning documents 48

4.4.1 Mobility Handbook provided by the Flemish Government 48 4.4.2 Guidelines for consultation on Urban Mobility Plans (CERTU) 49 4.4.3 Guidance on Local Transport Plans provided by the UK Department for Transport 49 4.4.4 The National situation in the Czech Republic 50

4.5 Evidence from interviews with urban mobility practitioners 50

4.5.1 Summary of findings from interviews with urban mobility practitioners 50 4.5.2 Characteristics of the sample 51 4.5.3 Approaches to evaluation 52 4.5.4 Pros and cons of different approaches to Sustainable Urban Mobility Planning, and lessons learnt 53

5 Comparison of the approach of QUEST, with that of other projects using TQM to develop audit tools 55

5.1 Quality management schemes 55

5.1.1 Total Quality Management in public administrations 55 5.1.2 TQM in the transport sector 56 5.1.3 Assessment process and the role of the auditor 60 5.1.4 Certification and benchmarking 62

6 Conclusions 64

6.1 The impact of approaches to plan evaluation on the achievement of European policy goals 64

6.2 The pros and cons of different approaches to evaluation 65

6.3 Conclusions regarding review of projects using TQM to develop audit tools 66

7 Recommendations for development of QUEST audit tool 67

7.1 Recommendations from the review of policies and legislation 67

7.2 Recommendations from the review of approaches to evaluation 68

7.3 Recommendations from the review of other projects using TQM Management 69

TABLES

Table 1 Components and elements of QM schemes in the transport sector ..............................................56

Table 2 Ladder of development in BYPAD,MaxQ and Mediate ..............................................................58

Table 3 Overview of recommendations based on success factors ..........................................................70

FIGURES

Figure 1 Scoring different elements in the Mediate project ...................................................................59

Figure 2: Three potential tracks of QUEST .......................................................................................61

APPENDICES

APPENDIX 1 - List of web-based databases consulted

APPENDIX 2 - Detailed inventory of relevant European legislation/policies

APPENDIX 3 - Detailed inventory of projects and documents relevant to SUMP evaluation

APPENDIX 4 - Suggested questionnaire to guide interviews with individuals involved in SUMP evaluation


03/01/2012 5 TTR

APPENDIX 5 - Detailed inventory of projects using TQM to develop audit tools

APPENDIX 6 - Full check-list of questions used by the BUSTRIP Project


03/01/2012 6 TTR

Executive Summary

The aims of the QUEST Project

The QUEST Project (Quality management tool for Urban Efficient Sustainable Transport), funded by Intelligent Energy Europe, aims to assist small and mid-sized cities in improving planning for sustainable urban mobility. To achieve this, QUEST will develop a tool for evaluating urban mobility policies. This tool will be based on the concept of Total Quality Management (TQM). This will involve working with 50 European cities on the improvement of their Sustainable Urban Mobility Plan (SUMP), or, where a SUMP has not been created, the policy document that is intended to fulfil similar sustainability objectives.

The Project‟s approach

QUEST will build on the experience and knowledge gained from current and previous projects and initiatives dealing with sustainable mobility planning. The project will also learn from the experiences of projects that have featured TQM in related fields, including accessibility and cycling policy. An important first step, therefore, has been the carrying out of a state of the art review of approaches to assessing and evaluating urban mobility planning. This exercise has involved reviewing three areas: 1. European policies and legislation, 2. Different approaches to evaluation, and 3. Initiatives that have used TQM for the purpose of developing audit tools. The immediate purpose of carrying out this wide-ranging review has been to create a digest of lessons learnt and knowledge gained as to the advantages and disadvantages of different approaches, which will inform the development of the QUEST Audit Tool in work package 4.

The review of European legislation set out to identify directives, policies and goals set by the European Commission, and to describe how they might influence the development of SUMPs at a local level. The objective of the review of European projects, initiatives and documents was to identify evidence for the pros and cons of different approaches to evaluation.

Both of these reviews were facilitated with the use of a standard template for the initial collection of information. This ensured that data were gathered and filed in a consistent and comprehensive manner, by a number of members of the project team. The template was designed to store data on the title of a project or document, a relevant web-link, the time-frame for the project or initiative, the relevance of the reference to SUMPs, information available on mobility planning at a local level, and lessons learnt that might influence the design of the QUEST Audit Tool. Each reference reviewed was further categorised according to its degree of relevance to the QUEST Project, and also by policy domain, (e.g. transport, environment, economic development, air quality, etc.).

Additional insights were gained on the pros and cons of different approaches to evaluation by means of a survey of urban mobility professionals. The review of other projects using the concept of TQM, such as Mediate, BYPAD, ECOMOBILITY SHIFT, MaxQ and ISEMOA, was informed partly by means of a review of relevant project documentation, but also to a considerable extent by project partners‟ experience of having worked on some of these projects.


03/01/2012 7 TTR

The main findings of the project

The key policy statements affecting urban mobility planning at a European level are the EC Action Plan on Urban Mobility and the EC White Paper on the Future of EU Transport Policy, (Roadmap to a Single Transport Area – Towards a Competitive and Resource-Efficient Transport System). Both the Action Plan (2009 - 2013) and the White Paper (2011 - 2050) are very new, and so it is difficult to make judgments as to their impact to date. The Action Plan has not yet reached its mid-point, and so has not yet been subject to a mid-term review, and the roadmap outlined in the 2011 White Paper sets out a policy vision that extends far into the future. Whilst it is to be anticipated that the financial incentives provided by the CIVITAS Plus and Intelligent Energy Europe (IEE) funding mechanisms will encourage local authorities to implement the White Paper‟s long-term policies, it is too early, at this stage, to state whether this has been the case.

Guidance on the evaluation and assessment of SUMPs, and similar planning documents, provided by the Commission is detailed, and targeted at providing practical assistance for mobility planning practitioners. There is also an emphasis on the dissemination of knowledge, through training and the showcasing of good practice, and this is supported by the availability of funding from the EU for both practitioner training and demonstration projects. A specific example of support for improving sustainable urban mobility planning is provided by the ELTIS portal, which is a networking platform through which local authority practitioners can informally benchmark, and learn from one another.

The EU also provides encouragement for linking energy & climate change and transport policy domains, and for transport & mobility issues to be incorporated into Sustainable Energy Action Plans (SEAPs). The White Paper‟s Roadmap towards a single transport area is also closely aligned to the Roadmap for Moving to a Low-Carbon Economy.

The detailed review of approaches to evaluation began with consideration of the findings of similar reviews carried out elsewhere, notably the one undertaken by the ELTIS+ Project, which were made available in July 2011. In addition, there was also a review made of a report by an Expert Working Group on Sustainable Urban Transport Plans (commissioned by DG Environment, in December 2004), and a report entitled “Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions“, (published by DG Environment, in October 2005).

The ELTIS+ State of the Art Review is a reference manual aimed at urban mobility planning professionals. It has an emphasis on professionals‟ training needs, the setting of targets that are both measurable and achievable, stakeholder participation, and the taking of an integrated approach to planning. The ELTIS+ review provides a definition of the key characteristics of a good SUMP, stating that its central objective is to ensure improvement according to the three main themes of environmental sustainability, social cohesion and economic efficiency. It further elaborates these themes to the important goals of ensuring access for all citizens, improving safety and security, and addressing specific environmental issues. The SUMP planning process is modeled as an eleven-step procedure, which is further broken down into 32 specific tasks. This set of guidelines is considered to be currently the single most authoritative document to support the development of SUMPs in Europe.

The remit of the Expert Working Group on Sustainable Urban Transport Plans was the design of procedures, instruments & measures and advice on the content of any future EC Directive on SUMPs, with recommendations graded according to whether they should be


03/01/2012 8 TTR

obligatory or advisory. Among the obligatory recommendations is the involvement of stakeholders at every stage of the planning process, and the use of a precisely defined set of indicators to guide evaluation. The Working Group goes further, by recommending that the European Commission should define a compulsory set of indicators.

Among the individual projects and initiatives reviewed in some detail were those which have contributed to laying the foundations for the evaluation of sustainable mobility plans. These included PILOT, the CIVITAS Evaluation Programme and EPOMM Plus. The PILOT Project highlighted the importance of monitoring and evaluation, and also of Cost Benefit Analysis, within the planning process, and developed tools and guidelines, such as the PILOT Evaluation Manual. PILOT provided the foundations for the ELTIS+ Project‟s recommendations for the minimum requirements for a good SUMP.

The CIVITAS Evaluation Programme is an important source of reference, having hosted many projects, including METEOR and GUARD, which coordinated evaluation activities in CIVITAS I and CIVITAS II, respectively. The QUEST Project also reviewed the early findings of the CIVITAS Plus project ARCHIMEDES. The main principles emerging from ARCHIMEDES have been the importance of engaging with stakeholders at every stage of the planning process, the definition of precise targets and indicators, and the provision of evaluation reports on an annual basis.

The EPOMM Plus Project focuses on the implementation of Mobility Management in Europe, and the dissemination of good practice. It also develops training initiatives, and uses a suite of evaluation and decision support tools previously developed by the Max Project. These tools include MaxExplorer (decision support), MaxEva (evaluation), MaxSumo (standardising evaluation), MaxQ (Quality Management) and MaxSem (behavioural and psychological modelling).

A common theme to emerge from this investigation has been the importance that many projects have placed on the setting of targets, in the early stages of planning, and the associated exercise of selecting appropriate indicators; this is something that is underlined in the PILOT Evaluation Manual. The ARCHIMEDES Project suggests that these targets should be measurable, quantifiable, relevant and realistic, whilst ELTIS+ states that these targets should be “SMART”, (i.e. Specific, Measurable, Achievable, Realistic, Time-bound).

One piece of advice to emerge from the interviews carried out with mobility planning professionals, in relation to the selection of targets and goals, is that care should be taken to limit the number of targets that are set. This is because of the issue that several respondents found with the cost of collecting data for the evaluation of performance against the targets that are set. It was suggested that the optimum number of targets or goals to be set lies in the range from 10 to 20. In connection with this, the QUEST interviews revealed a general consensus as to the importance of there being adequate financial resources for evaluation to be undertaken. The ELTIS+ recommendation is that the budget for evaluation should be at least 5% of the total budget set aside for the SUMP process as a whole, whilst the figure quoted by the CIVITAS II Project GUARD is 10%.

Engagement with stakeholders at all stages of the sustainable urban mobility planning process, including implementation and monitoring & evaluation, is another important theme to emerge from the review of projects and initiatives. The ARCHIMEDES Project has identified some specific lessons learnt in relation to the engagement of stakeholders in the process, The project advises that the engagement process should be inclusive, but proportional to the resources available to both the local authority and the stakeholders


03/01/2012 9 TTR

involved. Similarly, the MOBILIS Project advises that stakeholder involvement can be enhanced with the formation of a local “mobility group”, and the CO2NeuTrAlp Project goes further by describing an eight-stage co-operative Local Implementation Network.

The review of previous approaches to using Total Quality Management in the transport sector has focused on the projects BYPAD, Mediate and MaxQ. The BYPAD methodology is well established, and has been implemented in more than 100 cities in 21 countries. The Mediate Project concluded at the end of 2010, and so has only been tested in two locations (Lisbon and Flanders), whilst the MaxQ approach has been successfully used in Sweden. BYPAD, MaxQ and Mediate all use a structured methodology for evaluating the current situation of a planning system in a given city. In MaxQ, this results in a prioritised shortlist of promising measures. BYPAD and Mediate incorporate a further stage, consisting of the development of an Action Plan or Quality Plan, which emerges as a result of a process of consensus with stakeholders. This Action Plan confirms commitments to implementing improvements, by specifying a timetable for actions, and by allocating specific budgets and responsibilities. ECOMOBILITY SHIFT sets out to recommend improvements in each defined output category, (i.e. walking & cycling, public transport, mobility management, land use management and the use of private cars).

One conclusion of this review for the QUEST Project is that the Action Plan developed should consist of a clear improvement plan which prioritises feasible measures. This differs from the approach of ECOMOBILITY SHIFT, which requires actions to be complete. Since QUEST aims to support cities in actually making progress on sustainability, cities should have the flexibility to make choices, in conjunction with stakeholders, as to which measures will actually be implemented. These choices should be supported with a clear improvement programme which provides guidance on both long- and short-term solutions.

The recommendations made from the state of the art reviews carried out for work package 3 is as follows: -

Recommendations from the review of policies and legislation

SUMPs provide a framework which can help local authorities to meet the requirements of EU legislation, and so help to fulfill the wider goal of moving towards a more sustainable transport system in Europe. The purpose of reviewing such policies and legislation was to assess which might influence the design of the QUEST audit tool. The review set out to identify the most important aspects of directives and related guidelines which might be incorporated into the QUEST audit tool. These are as follows: -

• the theme of integration is very important to the whole concept of a SUMP, and this includes integration of the different levels of authority (i.e. European, National, Regional and Local Government levels), and across geographical borders. A key aspect of integration is that between policy domains, so that there is coordination and liaison between transport planners and land use planners, and those responsible for energy and climate change policies. The audit tool should therefore include evidence of such coordination in its appraisal of SUMPs, and, where it is deemed to be deficient, encourage greater policy integration. For example, the tool should be used to encourage links with local Sustainable Energy Action Plans (SEAPs), and should look for evidence of awareness of land use planning policies, which might include plans for neighbourhood renewal, plans for housing projects, etc. Where there appears to be insufficient consideration of land use policies, the auditor can raise awareness of the resources such as MaxLupo, a tool designed to facilitate the integration of Mobility Management and land use planning activities, and one of a suite of such tools


03/01/2012 10 TTR

developed by the MAX Project. MaxSumo and MaxEva are tools specifically designed to assist local authorities with monitoring and evaluation tasks, and local authorities should be made aware of their availability, if they are not already aware. The audit should also encourage links between SUMPs and Air Quality Plans (AQPs).

• another important characteristic of a SUMP is that it should address the three broad policy objectives of environmental protection, economic development and social cohesion. Certainly, there was evidence from the QUEST Project‟s interviews with mobility planning professionals that too much emphasis should not be placed on environmental objectives and indicators, since the advantages of sustainable mobility planning in terms of enhancing access, and the quality of life, for citizens, and the accompanying economic benefits, are seen to provide a more compelling argument for persuading politicians and the public at large of the value of sustainable urban mobility planning. The extent to which a plan considers all of these policy dimensions should be an important aspect of the audit process.

• the audit tool should also take account of the extent to which the local authority engaged in sustainable urban mobility planning takes advantage of the resources, facilities and advice that are made available by the European Commission. The advantage of including this issue in the audit is that, where there is scope for an increase in the take-up of such help and advice, then the QUEST Project can provide assistance as to what is available, and how it might be obtained. An example of an important facility is the ELTIS Urban Mobility Portal, which provides guidance materials, the opportunity for the exchange of best practice, and advice on the training of Mobility Management personnel. (This portal was, however, only relaunched in March of 2011, so the main objective of including its use in the audit might be the raising of awareness about the facility).

• SUMPs and similar documents should be assessed in terms of whether they are comprehensive in dealing with freight, as well as passenger, issues. This is particularly important in the context of “last kilometre” deliveries in urban areas.

Recommendations from the review of approaches to evaluation

• One of the most common areas for good practice found with evaluations from elsewhere concerns the selection of targets and indicators on which subsequent evaluations are based. The precise detail of the advice offered varies between the different guidelines, projects and initiatives, but the important principles that should be taken on board during the creation of the QUEST audit tool are that: -

- Precise targets and objectives, and a set of appropriate indicators that will be used to effect the evaluation, should be defined at the beginning of the sustainability planning process

- The choice of indicators should maximise the probability that the evaluation process should be effective and feasible to complete within envisaged timescales and available budgets. Several versions of formal guidance were found in the literature – including the rule of thumb that targets should be “SMART” (i.e. Specific, Measurable, Achievable, Realistic and Time-bound) – but the important principle from the point of view of auditing is that targets and indicators should be suitable in order to enable the evaluation process to be completed, given local considerations of resource and data availability.


03/01/2012 11 TTR

• Proper involvement of stakeholders in the planning process is an issue that is widely discussed in the literature. In particular, an audit should assess the extent to which stakeholders are engaged at each stage of the process, from planning, through to implementation, and monitoring and evaluation. The stakeholders invited to take part in the process should be representative of all interested parties, and include both individuals and representatives of organisations. They should be adequately briefed as to the remit for their involvement, the scale of which should be appropriate to their own resources and availability.

• The issue of who is involved in the evaluation process is an important one. Ideally, evaluation should be carried out by a body that is external to, (i.e. independent from), the main process. The audit should also consider the extent to which the evaluator(s) have the necessary skills, resources and capacity for carrying out the evaluation.

• Another key element in evaluation is ascertaining the extent to which plans are in place to ensure that adequate financial resources are made available, to ensure both implementation and evaluation of the Plan. This is important, given the consensus that data collection for the purpose of evaluation can be a costly and resource-intensive activity. The auditor should be able to provide advice on funding that might be available from EU sources, such as Structural and Cohesion Funds, for larger investments, (for more sustainable vehicles and infrastructure, etc.).

• An important aspect of the planning process is the extent to which local politicians are involved in this process, as this is felt to be key to ensuring successful implementation. Political approval for the goals and priorities of the plan can help to ensure that adequate financial resources are dedicated to both implementation of the planned policy interventions, and the necessary monitoring and evaluation activities, (see above). The audit procedure should asses the extent to which there is effective political involvement in the process.

• Evaluation should be part of a cyclical planning process, with a schedule in place to ensure that it is repeated at least annually. The results of evaluations should be taken into account in subsequent planning and implementation phases, before being re-evaluated on at least an annual basis.

• The audit should ensure that there is evidence of the results of each round of evaluation being disseminated to the public domain, so that they might inform the public debate on sustainability issues.

• The process evaluation aspect of an audit should consider, as a minimum, the comprehensiveness of the plan (in terms of the range of aspects of sustainable mobility that it addresses), its efficiency in terms of its use of available resources, and its effectiveness, (in the context of delivering planned outputs).

Recommendations from the review of other projects using Total Quality Management

• QUEST needs to create an attractive product which helps cities to make progress in the development and implementation of a sustainable mobility policy. QUEST must find the right balance between simplicity of use, and effectiveness and credibility.

• Stakeholders involvement is a crucial aspect of TQM, in contrast with regular auditing. By involving local stakeholders in the assessment and the improvement plan, it is more likely that improvements will be put on the agenda and really be implemented. Moreover, there is a


03/01/2012 12 TTR

bigger chance that the proposed changes are regarded as important and appreciated by the end users. Therefore QUEST should involve key-stakeholders in the whole process of QUEST.

• The QUEST Action Plan should consist of a clear and prioritised improvement plan, where precise actions are agreed in collaboration with stakeholders.

• QUEST should focus on certification of cities‟ efforts in sustainable mobility planning. Cities should be able to choose use of their certification status, (i.e. whether it is used externally).

• QUEST should also focus on its persuasiveness in providing clearly defined measures, based on the agreement of key stakeholders, in order to demonstrate the added value of its involvement. By the same token, QUEST should prove its effectiveness at a national level, helping participating cities to secure additional transport funding. Looking further ahead, QUEST should try to gain recognition from an accreditation body such as ISO.

• Sound communication on the methodology and the effective training of auditors are both crucial. The auditor has to set up cooperation at the city level, and guides the city through the self assessment process, assuring its quality. QUEST must clearly define the skills and responsibilities of the auditor.


03/01/2012 13 TTR

1 Introduction

The QUEST Project will develop a tool for evaluating urban mobility policies, and so assist local authorities to improve planning for sustainable urban mobility. This will be done with the involvement in the project of 45 European cities and towns. QUEST will build on experience of existing quality management tools for urban transport and environment. At the end of an audit, each city will receive a certificate recognising its efforts in sustainable urban mobility planning. Each city will also be given a tailor made Action Plan to support them in progressing towards a more sustainable urban transport system. This will be informed by the present quality of urban mobility policy. QUEST distinguishes three types of improvement programme, relating to 'starter', 'climber' and 'champion' cities.

QUEST is divided into a series of work packages (WPs), including work package 3, which focuses on the state of the art of urban mobility assessment. This work package reviewed: -

how EU directives, policies and goals have influenced local mobility planning

different approaches to evaluation of mobility plans at the local level

projects and initiatives using the concept of Total Quality Management (TQM) to create a tool for auditing urban transport systems.

This report is the deliverable from work package 3. It considers the state of the art of urban mobility assessment and includes recommendations for the approach to be taken by QUEST.

1.1 A note on terminology

The terms „Sustainable Urban Mobility Plans‟ (SUMPs) and „Sustainable Urban Transport Plans‟ (SUTPs) are widely used across Europe in the context of urban mobility planning. The QUEST project, including this report, generally utilise the term „Sustainable Urban Mobility Plans‟ and its abbreviation when referring to urban mobility planning. An exception is where references are made to projects or documents which have used the term „Sustainable Urban Transport Plans‟ or its abbreviation.


03/01/2012 14 TTR

2 Methodology

Work package 3 was led by Transport & Travel Research Ltd (TTR), with support from Ligtermoet & Partners (L&P), Koucky & Partners (K&P), Centrum Dopravniho Vyzkumu (CDV), European Institute for Sustainable Transport (EURIST) and Tritel. This work package commenced in month 2 (June 2011) and finished in month 6 (October 2011). It comprised three tasks.

2.1 Task 3.1: State of the art of goals and strategies at a European level

This task was led by TTR. Its objective was to identify directives, policies and goals set by the European Commission, and to describe how they might influence the development of SUMPs at a local level. The work comprised a review of EU policies, directives and goals in support of thematic strategies directed towards economic development, environment, climate change and public health. It was considered how such legislation and policies influenced mobility planning at a local level.

Relevant EU legislation and policies were identified in various ways: -

from existing knowledge and experience of staff working on the task

from searches of web-based databases, based on a list of fora and online resources included in the Description of the Action (DoA)1, plus some additional websites identified by staff. The list of websites consulted appears as Appendix 1

review of certain items of legislation and policy documents revealed additional pertinent references

A review was completed for each reference, using a standard template to ensure a consistent and comprehensive approach. The details recorded in the template were: -

official title

EU reference (where specified) and weblink

timeframe

relevance to sustainable urban mobility planning

influence on mobility planning at a local level

classification by: -

o theme, i.e. development, environment, climate change (including low carbon futures), health and public safety, and transport

o relevance, i.e. limited relevance, of some relevance, relevant.

The full set of completed review templates is contained in Appendix 2. This appendix also provides details of some sources found, but not considered further due to lack of relevance. The details reported in Appendix 2 informed preparation of section 3 of this report, review of European legislation/policies relevant to SUMPs, as well as the conclusions section.

2.2 Task 3.2: State of the art of approaches to assessing sustainable urban mobility plans

Task 3.2 was the largest component of work package 3. It was led by TTR, with input from K&P, CDV, EURIST and Tritel. The objective of this task was to review European projects, initiatives and documents in order to identify evidence for the pros and cons of different approaches to evaluation.

1 Annex 1 to the Grant Agreement for QUEST.


03/01/2012 15 TTR

Relevant projects, initiatives and documents were sourced in the following ways: -

from a list of projects, initiatives and documents incorporated in the DoA

from existing knowledge and experience of staff involved in the task

from searches of web-based databases. (See Appendix 1 for the list of websites consulted)

review of various items of EU legislation/policy documents within task 3.1 highlighted additional references that were relevant to investigate (within task 3.2)

review of certain sources within task 3.2 highlighted additional references that were relevant to consider.

A review was conducted for each source, again utilising a template to ensure a consistent and comprehensive approach. The details recorded in the template were: -

official title

weblink(s)

timeframe/publication date

funding programme (where specfied)

partners and other stakeholders involved/authors

remit

approach to evaluation

lessons learnt regarding evaluation

reviewer‟s notes to inform recommendations section of this report

relevance rating, i.e. limited relevance, of some relevance, relevant.

The full set of completed review templates is contained in Appendix 3. The appendix also gives details of some references identified, but not pursued further due to lack of relevance or availability. Additionally, contact was made with a selection of staff at organisations that implement SUMPs/individuals who are evaluation experts. This was so QUEST could obtain views on the impact of sustainable urban mobility planning on the ground, and not just rely on periodic project outputs. A series of 20-minute telephone interviews was held, focusing on some key questions about evaluation. A form was designed with a list of suggested questions, (see Appendix 4). Seven individuals were interviewed, representing a fair geographical spread.

The reviews reported in Appendix 3, and the interviews conducted, fed into preparation of section 4 of this report, (the review of approaches to evaluation). They also informed the report‟s conclusions and recommendations for development of the QUEST methodology/audit tool.

2.3 Task 3.3: Review of projects using the concept of TQM to develop an audit and self-certification tool

This task was led by L&P, with contributions from K&P, CDV, EURIST and Tritel. The work consisted of reviewing projects and initiatives with a similar approach to QUEST, i.e. using TQM to develop a methodology for auditing urban transport systems or aspects of them.

Relevant projects and initiatives were sourced as follows: -

from a list of projects included in the DoA

from existing knowledge and experience of staff working on the task2.

2 There was some overlap between tasks 3.2 and 3.3, i.e. a few projects were pertinent to both SUMP evaluation and use of TQM to audit urban transport systems. In these cases, the leaders of tasks 3.2


03/01/2012 16 TTR

A review was completed for each source, again using a standard template to achieve a consistent and comprehensive approach. The details reported in the template were: -

official title

weblink(s)

timeframe

funding programme (where specfied)

partners and other stakeholders involved/authors

remit

description of the auditing process

scope of implementation

role of the auditor

arrangements for labelling or certification

lessons learnt applicable to development of the QUEST tool

reviewer‟s notes to inform recommendations section of this report

relevance rating, i.e. limited relevance, of some relevance, relevant.

The full set of completed review templates appears in Appendix 5. Some sources were found, but not investigated further, due to a lack of relevance. The details reported in Appendix 5 informed preparation of section 5 of this report, (review of projects using TQM to develop audit tools), as well as the report‟s conclusions and recommendations.

2.4 General comment

One factor affecting the relevance of references identified within tasks 3.1-3.3 is their age. Most sources reviewed date from within the last five years. It was found that certain European legislation/policy documents/projects had been overtaken by later sources. However, it should also be noted that some of the projects reviewed are ongoing. This means that the reviews, including assessment of relevance, had to be based on information available from the projects at the time of review (June-September 2011). It is likely that relevant outputs will be generated subsequently from these projects, which QUEST work package 3 will be unable to take into account.

and 3.3 liaised to agree under which task the project should be reviewed, thereby avoiding duplication of work.


03/01/2012 17 TTR

3 Review of European legislation/policies relevant to SUMPs

3.1 Summary of findings from review of European legislation and policies

The key policy statements at a European Union level that affect urban mobility planning are: -

EC Action Plan on Urban Mobility (2009 - 2013)

EC White Paper on the Future of EU Transport Policy, Roadmap to a Single Transport Area – Towards a Competitive and Resource-Efficient Transport System (2011 - 2050).

The key content of these policy documents is described in sub-sections 3.2 and 3.3, respectively.

Other legislation and policy documents relevant to the subject area are described in subsequent sub-sections, but many of these have been superseded by these two more recent documents. Both the Action Plan on Urban Mobility and the White Paper are very new, and so it is difficult to make judgments as to their impact, either to date, or in the future. The Action Plan has not yet reached its mid-point, and so has not yet been subject to a mid-term review. Similarly, the Roadmap outlined in the 2011 White Paper sets out a policy vision that extends far into the future. Whilst it is to be anticipated that the financial incentives provided by the CIVITAS Plus and Intelligent Energy Europe (IEE) funding mechanisms will encourage local authorities to implement the White Paper‟s long-term policies, it is too early, at this stage, to state whether this has been the case. The same also applies to more detailed recommendations, such as the use of a European urban mobility scoreboard to encourage policy implementation.

As described in the following sub-sections, guidance provided by the Commission is detailed, and targeted at providing practical assistance for practitioners engaged in implementing sustainable mobility action plans. Furthermore, Action 2 of the current EC Action Plan aims to raise awareness of the funding opportunities that are available. The Action Plan was derived from a thorough and wide-ranging process of consultation with practitioners who are responsible for implementation, so the advice and guidance provided should be tailored to the needs of these practitioners. The advice given is also very practical, and directly relates to SUMPs. There is also an emphasis on the dissemination of knowledge, through training and the showcasing of good practice, and this is supported by the availability of funding for both practitioner training and demonstration projects. A specific example of support for improving sustainable urban mobility planning is provided by the ELTIS portal, which is a networking platform through which local authority practitioners can informally benchmark, and learn from one another.

A common thread that runs through policies and guidance relating to SUMPs is the issue of integration. This refers to the importance of the integration of policies from different sectors (most importantly transport and land-use planning), at different levels (i.e. European, National, regional and local) and across geographical boundaries. There is encouragement for linking energy & climate change and transport policy domains, and encouragement also for transport & mobility issues to be incorporated into Sustainable Energy Action Plans (SEAPs).

The new White Paper‟s Roadmap towards a single transport area is closely aligned to the Roadmap for Moving to a Low-Carbon Economy. In summary, it uses three broad mechanisms for reducing the environmental impact of the transport sector: increasing the efficiency with which current transport systems provide mobility, encouraging the use of more sustainable vehicles, and promoting modal shift to more sustainable modes of transport. The White Paper includes specific targets for emissions reduction, specifically a 60% decrease in


03/01/2012 18 TTR

CO2 from transport by 2050. This goal is supported by the Resource Efficient Europe Initiative‟s target of an 80% overall domestic reduction in Greenhouse Gas emissions (GHG) by 2050, compared with 1990 levels.

The current EC Thematic Strategy on the Urban Environment, which covers environmental issues in general, also includes a section on transport. It lays emphasis on the importance of both stakeholder consultation, and the training of local authority personnel who are charged with the task of implementing sustainable mobility policies.

This section also describes the contribution made by the EU‟s Strategy for Sustainable Development, the Adapting to Climate Change White Paper, the 2011 Energy Efficiency Plan, the current Ambient Air Quality Directive, the Action Plan for the Deployment of Intelligent Transport Systems (ITS) in Europe, and current policies on the Promotion of Clean and Energy-Efficient Road Transport Vehicles.

3.2 EC Action Plan on Urban Mobility (2009 - 2013)

The Action Plan on Urban Mobility was adopted on the 30th of September 2009, and follows the 2007 Green Paper “Towards a New Culture for Urban Mobility”. It also follows the EC‟s Communication document “A Sustainable Future for Transport: Towards an Integrated, Technology-led and User Friendly System”, which calls for coordination of actions at a local, regional and national level. The detailed recommendations of the Action Plan are the result of feedback received during the stakeholder consultation exercise that was initiated by the aforementioned Green Paper.

This document is directly relevant to the work of the QUEST Project, in as much as it provides a comprehensive package of support for helping local, regional and national authorities to achieve urban mobility sustainability targets. The Action Plan makes specific reference to Sustainable Urban Mobility Plans (SUMPs).

The Communication emphasises the importance of local authorities taking measures to encourage sustainable mobility, but also emphasises the importance of there being a framework of policy and legislation at EU level to support them in achieving this. The EU‟s approach is very much in line with the principle of subsidiarity, in that it pledges the necessary support for local authorities in creating effective Sustainable Urban Mobility Plans (SUMPs), whilst acknowledging that the detail of such SUMPs will necessarily be dictated by local conditions and resources. The EU provides support in the form of funding, and in encouraging the exchange of information on best practice. It proposes both short- and medium-term actions which authorities are invited to commit to on a voluntary basis. This document also claims that, at a higher level, the EU can facilitate the creation of more sustainable transport across the Single Market area by encouraging interoperability, and by providing uniformity in relevant standards, rules and technologies. The role of the authorities is to make funding available, and to provide a regulatory framework and a planning function, at a local, regional and national level.

The primary challenges addressed by the Plan are the need to make transport more sustainable in the environmental contexts of air pollution, CO2 emissions and noise, and the need to improve economic competitiveness by reducing traffic congestion. The document also acknowledges that other important priorities should be considered, such as public health, the needs of older and disabled people and their families, and social cohesion in general.

The stated aims of the Action Plan include: -

promoting integrated policies to deal with the complexity of transport systems

optimising urban mobility to encourage effective integration between different transport networks, and

sharing experience and knowledge.


03/01/2012 19 TTR

The 20 numbered Actions are categorised within one of six themes: -

Theme 1. Promoting Integrated Policies

Theme 2. Focusing on Citizens

Theme 3. Greening Urban Transport

Theme 4. Strengthening Funding

Theme 5. Sharing Experience and Knowledge

Theme 6. Optimising Urban Mobility.

The following Actions are of most relevance to the creation and evaluation of SUMPs: -

Action 1 (Theme 1): Accelerating the take-up of sustainable urban mobility plans. One means adopted by the European Commission for achieving this is the linking of energy and climate change policies with transport. For example, the EU encourages transport and mobility issues to be included in the Sustainable Energy Action Plans (SEAPs) prepared by cities participating in the Covenant of Mayors. This is covered by guidelines on the development of SEAPs. As of June 2011, five SEAPs had been approved, and 489 were under evaluation throughout Europe. This Action is also supported by the ELTIS Urban Mobility Portal (Intelligent Energy Europe Programme (IEE)), which was relaunched in March 2011. The Commission‟s pledge in the short term is to provide support for SUMPs covering freight and passenger transport in urban and peri-urban areas, in the form of guidance materials, the promotion of best practice exchange, the identification of benchmarks and support for education for urban mobility professionals. Longer-term support is defined in general terms as being through incentives and recommendations.

Action 2 (Theme 1): Sustainable urban mobility and regional policy. This Action is evidence of the Commission‟s pledge to increase awareness, among authorities involved in SUMPs, of funding that is available from the Structural and Cohesion Funds, and from the European Investment Bank. The Commission will achieve this by emphasising the links between urban transport and the trans-European transport network, and between sustainable urban mobility measures and regional policy objectives.

Action 8 (Theme 2): Campaigns on sustainable mobility behaviour. The Commission recognises the importance of education, information and awareness-raising in creating a culture that encourages sustainable urban mobility, and so will support such campaigns at all levels. An example of on-going support is European Mobility Week, and the related award scheme. Consideration is being given to introducing a special award relating to the adoption of SUMPs.

Action 14 (Theme 4): Optimising existing funding sources. The Structural and Cohesion Funds, mentioned in Action 2, are the main source of funding for investment in infrastructure and rolling stock, which is crucial for creating the conditions for more sustainable urban mobility. The Action Plan describes how the Commission will, in addition, fund RTD and Demonstration activities within the sustainable urban mobility priority area of FP7‟s “Transport” Theme. Action 14 also includes the continuation of EC support for Intelligent Energy Europe‟s STEER Programme, the CIVITAS Initiative, URBACT and the Information & Communication Technologies Policy Support Programme. CIVITAS funds projects which demonstrate sustainable urban mobility locally. Since 2002, through CIVITAS I, CIVITAS II and CIVITAS Plus, €180m has been provided for 58 European cities, and an indicative budget of €18m is being considered for proposals for CIVITAS Plus II funding. The indicative budget for the 2011 round of proposals under the STEER Programme is some €12m.

Action 16 (Theme 5): Upgrading data and statistics. To facilitate planning for sustainable urban transport and mobility, the Commission undertook to fund a study on


03/01/2012 20 TTR

how the collection of relevant data could be improved. This is due to a perceived lack of data and statistics in this area.

Action 17 (Theme 5): Setting up an urban mobility observatory. Related to Action 16, the Commission pledged to support urban transport professionals by providing a “virtual platform” which would enable the sharing of data and statistics, the exchange of best practice, and the ready availability of training and educational materials. This has been realised in the form of the ELTIS (European Local Transport Information Service) portal, which provides practitioners with an on-line resource that is specifically focused on SUMPs.

Action 19 (Theme 6): Urban freight transport. The issue of the place of freight transport in SUMPs is not overlooked in the Action Plan, in particular the importance of maximising the efficiency of “last mile” delivery. Action 19 aims to provide assistance on how freight transport can be incorporated in local plans and policies, and how the flow of goods can be better managed and monitored. This aspect of sustainable urban mobility planning is linked to the Freight Logistics Action Plan. One output from this Action was the “Conference on Urban Freight Transport and Logistics: Innovative and Sustainable Solutions for Europe”, which was held in Brussels in November 2010.

The ELTIS portal, mentioned under Action 17, describes itself as “Europe's main portal on urban mobility”. It facilitates the exchange of information, knowledge and experience regarding urban mobility, and is aimed at individuals working in transport. ELTIS also provides access to a separate website about SUMPs, at http://www.mobilityplans.eu/index.php.

The portal provides the following: -

a regular round up of the latest local, regional and European transport news

a listing of forthcoming events related to urban transport

case studies of examples of successful urban transport strategies and initiatives, which can provide urban mobility professionals with insights into success factors

practical tools, including guides and handbooks, to support urban transport professionals in their work

a range of other services, including an e-forum, European policy reviews, information about European funding sources, transport statistics, a photo library and training materials.

Transport professionals can submit news items, event details, case studies, training resources and job offers, for inclusion on the ELTIS portal.

The SUMP website provides information about the benefits of SUMPs, and guidelines on the different stages of developing and implementing SUMPs. These guidelines include support with such practical issues as assessing the baseline, setting targets, developing measures, allocating responsibilities and funding, and evaluation. The guidelines also incorporate examples of good practice.

Among the 20 Actions that are itemised in the Action Plan, those which have less relevance to the SUMPs at a local level, but nevertheless have an important role to play in facilitating more sustainable urban mobility, are “Action 5 (Theme 2): Improving accessibility for persons with reduced mobility”, “Action 6 (Theme 2): Improving travel information” and “Action 20 (Theme 6): Intelligent transport systems (ITS) for urban mobility”. These three Actions all address issues which make more sustainable modes of transport easier and more pleasant to use for all members of the travelling public. In relation to Action 20, the Commission pledges to provide assistance on ITS applications for urban mobility, as a complement to the ITS Action Plan. This will include consideration of support for electronic ticketing and

http://www.mobilityplans.eu/index.php


03/01/2012 21 TTR

payment, the use of smart cards in urban transport, the interoperability of ticketing and payment systems, traffic management, travel information and travel demand management.

3.3 EC White Paper on the Future of EU Transport Policy, Roadmap to a Single Transport Area – Towards a Competitive and Resource-Efficient Transport System (2011 – 2050)

This White Paper was published in March 2011. It has succeeded the existing EC White Paper on Transport, “European Transport Policy for 2010: Time to Decide”, and the EC Communication “A Sustainable Future for Transport: Towards an Integrated, Technology-Led and User-Friendly System”, which was the first step in defining future European transport policy. The White Paper also complements the “EC Roadmap for Moving to a Low-Carbon Economy in 2050”.

The earlier White Paper, which included an action programme up to 2010, comprised some 60 measures to be taken at a European Commission level, but none of these related to SUMPs. The most relevant section of this earlier document was a sub-section, (IV of Part 3), relating to the rationalisation of urban transport. This emphasised the importance of placing users at the heart of transport policy, and promoted good practice.

The EC Communication “A Sustainable Future for Transport: Towards an Integrated, Technology-Led and User-Friendly System”, published by the DG for Energy and Transport, reflects upon future directions and priorities for transport policy. The content of this Communication is based upon a specific evaluation study of European transport policy, the findings from focus groups, output from “Transvisions” study (which considered some possible low-carbon scenarios for transport), and the results of a stakeholder consultation exercise that featured a high-level stakeholder conference held in March 2009. The stated objective of the document was to stimulate further debate, to identify policy options that might be considered in the later 2010 White Paper.

The high-level EC Roadmap for Moving to a Low-Carbon Economy is part of the Europe 2020 Strategy for a Resource Efficient Europe, and comprises a plan to meet the long-term target of reducing 1990 levels of domestic emissions by 80%-95%, by 2050. Transport is one of a number of sectors addressed by this roadmap. Means for the transport sector to make the transition to a low-carbon economy include improved operational efficiency through better information and communication, pricing schemes, improved vehicle efficiency and the expansion of public transport.

The new White Paper calls for a reduction in CO2 emissions from transport of at least 60% by 2050, but a key element of the overall strategy is ensuring the growth of the transport system and the maintenance of mobility levels whilst emissions reduction targets are achieved. The document describes various targets relating to different types of journey, including those within cities. This includes a proposal for a substantial shift to cleaner cars and fuels, public transport and active travel, entailing a 50% shift away from conventionally fuelled cars by 2030, phasing them out in cities by 2050. Other goals specified include the establishment of a framework for a European multi-modal transport information and payment system, and the achievement of “close to zero” fatalities in road transport by 2050, with road casualties in the EU halved by 2020.

The White Paper identifies the need for mixed urban strategies to reduce congestion and emissions, involving land-use planning, pricing schemes, more efficient public transport services, infrastructure for non-motorised modes and charging / refuelling facilities for clean vehicles. The document stipulates that cities above a certain size should be encouraged to develop urban mobility plans which bring all of these elements together. These plans should be fully aligned with integrated urban development plans.


03/01/2012 22 TTR

The White Paper presents a set of measures to be used for achieving specific targets. Some of these are directly relevant to the creation and evaluation of SUMPs at a local level; for instance: -

initiatives for sustainable urban mobility, such as Smart Mobility Partnerships and Demonstration Projects, which include initiatives on urban road pricing and access restrictions

the introduction of procedures and financial assistance, at a European level, for urban mobility audits and urban mobility plans that are undertaken by local authorities on a voluntary basis.

the setting up of a European urban mobility scoreboard based on common targets

consideration of the possibility of introducing a mandatory approach for cities of a certain size, according to national standards, and based on EU guidelines

the linking of regional development and cohesion funds to cities and regions that have submitted current, independently validated urban mobility performance and sustainability audit certificates

examination of the possibility of a European support framework for progressive implementation of urban mobility plans in European cities

development of integrated urban mobility in a possible Smart Cities Innovation Partnership

encouragement for large employers to develop corporate mobility management plans.

3.4 Other important legislation and policy documents

3.4.1 EC Communication: A Resource-efficient Europe – Flagship Initiative under the Europe 2020 Strategy (2010 – 2020)

“Europe 2020” is the European Union's high-level strategy for smart, sustainable and inclusive growth for the ten-year period up to and including 2020. The strategy has five headline targets at a European level; one of these relates to climate change and energy, and it has several sub-targets. The most relevant of these sub-targets is for greenhouse gas emissions (GHG) to be 20% lower than they were in 1990, by 2020. (A desire is expressed for GHG emissions to be reduced by up to 30%, if conditions allow). The sub-targets are translated into national targets for each EU Member State, reflecting the level of contribution each is able to make to the overall EU reduction. National targets are set in each country‟s Annual National Reform Programme.

The Europe 2020 strategy refers to the use of existing tools to boost growth. These include investment via the European Regional Development Fund, European Social Fund and Cohesion Fund to support “smart transport”, among other activities. The overall strategy comprises several flagship initiatives, including Resource Efficient Europe, under the category of Sustainable Growth, which aims to work towards a resource efficient, low carbon economy. The objective, here, is to provide a long-term framework for policy actions, including actions relating to transport. The key components of this long-term framework are a series of co-ordinated roadmaps. The roadmaps most relevant to SUMPs are the “Roadmap for Moving to a Low-Carbon Economy” and “Roadmap to a Single Transport Area - Towards a Competitive and Resource Efficient Transport System (EC White Paper)”, which have both been described in the previous section.

The essential link between this high-level strategy document, and local and regional authorities, is the Europe 2020 Monitoring Platform. This Platform was set up, with more than 150 regions and cities involved, by the European Union‟s Committee of the Regions, an advisory body representing regional and local authorities in Europe. This enables members


03/01/2012 23 TTR

to contribute to the EU debate on Europe 2020, and facilitates the exchange of good practice between local and regional policy makers. The Monitoring Platform provides,

an interactive website: http://cor.europa.eu/europe2020

policy focused workshops with Europe 2020 members, experts and observers

an annual questionnaire for collecting data from members on the implementation of the Europe 2020 agenda.

Another on-line networking platform that is provided by the European Commission is Regio Network 2020: https://webgate.ec.europa.eu/regionetwork2020/. This is also intended to help regional and local authorities to contribute to achieving Europe 2020 goals.

Local transport projects may receive support from the funds referred to above. For example, the European Regional Development Fund (ERDF) finances the INTERREG Initiative (Innovation and Environment Regions of Europe Sharing Solutions). INTERREG IVC, which is part of the current initiative, which covers the period 2007-13, has provided €15m of funding for seven projects related to sustainable urban transport.

3.4.2 EC Thematic Strategy on the Urban Environment (2006 onwards)

As the title of the document suggests, this Strategy considers the environmental challenges of Europe‟s urban areas in general, so has a wider scope than the domain of transport. However, transport related issues feature prominently, with the discussion featuring air quality, traffic levels & congestion, greenhouse gas emissions and urban sprawl. SUMPs are featured in as much as the Strategy contains some strong recommendations for local authorities to develop and implement such Plans, and it proposes some key actions relating to the production of guidance to assist in this process. The Strategy notes the importance of long-term transport planning, in order to ensure that financial arrangements are in place for procuring the necessary infrastructure and vehicles for the creation of an attractive and efficient public transport network, and to ensure coordination with land-use planning. It is noted that transport plans should take account of safety & security issues, access to goods & services, air pollution, noise levels, greenhouse gas emissions and energy consumption, and that both passenger and freight transport should be included.

A particularly strong recommendation is that effective consultation with all stakeholders should be a key feature in the implementation of environmental legislation at a local level, and the SUMP is the appropriate vehicle for such consultation. There is also a response to calls from “many local authorities” for training. This refers to training for transport personnel in the specific skills that are required for adopting an integrated approach to management. Such training should cover environmental legislation, effective public participation, changing people‟s choices & behaviour, and cross-sector cooperation. The Strategy specifies face-to-face training as being the most effective method of learning, and identifies exchange programmes for local authority officials, possibly supported by the LIFE+ Regulation, as being an option.

The Strategy has something to say on the relationship between the local authorities and the European Union, emphasising that it is the former that have a decisive role in improving the urban environment. It states that local authorities should be encouraged to adopt increasingly integrated approaches to managing the urban environment, guided by long-term and strategic action plans. It adds, however, that action needs to be taken at European, national and regional levels, and that the EU has a role to play in supporting Member States and local authorities by promoting best practice and encouraging networking between cities.

The document concludes by quoting two examples of good practice – these are from Copenhagen and Nantes. The former relates to the City of Copenhagen‟s adoption of a “Local Agenda 21 Strategy”, which is a requirement under Danish law. This involves the environmental management system for the entire city, together with more specific strategies

http://cor.europa.eu/europe2020

https://webgate.ec.europa.eu/regionetwork2020/


03/01/2012 24 TTR

for traffic, waste, CO2 and risk management. The City reports the identification of early benefits in relation to air quality, greenhouse gas emissions, energy use and ecological footprint. An additional positive impact has been improved social inclusion and better community relations as a result of work carried out with ethnic minorities on the development of local solutions to environmental issues.

The example from Nantes features the conurbation‟s “Plan de Déplacements Urbains (PDU)”, which is a sustainable urban mobility plan that is required under French law. This Plan sets out to meet statutory targets that have been set for controlling demand for the use of private cars (specifically, a reduction to 50% of all journeys made, from a starting-point of 62% in 2002), reducing CO2 emissions, air pollutants and noise, managing car parking supply, promoting cycling and supporting the development of travel plans by private businesses and public bodies. The document lists progress that has been made in Nantes to date, in the form of a completely renewed and clean bus fleet, extensions to the public transport network, integrated ticketing for all public transport modes, voluntary travel plans developed by major employers in the city, improvements to the city‟s cycling network, and the creation of Park and Ride services.

3.4.3 Mainstreaming Sustainable Development into EU Policies – 2009 Review of the EU Strategy for Sustainable Development (2009 onwards)

This Communication consists of a second progress report on the Sustainable Development Strategy of the European Union (EU SDS), in response to a request for such a report from the European Council in December 2007. The Review reports EU policy to have progressed in all of the Strategy‟s priority areas, but also notes the persistence of some unsustainable trends, such as a continued rise in energy consumption within the transport sector. Whilst there is no specific reference to SUMPs in this document, reference is made to recent policy developments, including the launch of new actions as a follow-up to the Green Paper “Towards a New Culture for Urban Mobility”, published by the DG for Energy and Transport in 2008.

3.4.4 EC White Paper: Adapting to Climate Change – Towards a European Framework for Action (2009 - 2012)

This White Paper sets out an EU-wide strategy for reducing the vulnerability to climate change of Member States. It builds on consultation launched in 2007 by the Green Paper “Adapting to Climate Change in Europe”. Four actions are emphasised: 1. Building a knowledge base; 2. Integrating adaptation into key policy areas; 3. Delivering adaptation using a combination of policies; 4. Encouraging international cooperation, (although there is an emphasis on cooperation at all levels, including the EU and national, regional and local authorities). Issues of water scarcity and the impact on farming are discussed, but there is no specific reference to urban mobility.

3.4.5 Renewed EU Strategy for Sustainable Development (2006 onwards)

This Renewed EU Strategy was produced following a review in 2005 of the 2001 EU SDS, and is an over-arching Strategy on sustainable development for all EU policies. Sustainable transport is one of seven key challenges listed in this Strategy. Among the various actions proposed is that local authorities should develop and implement SUMPs, taking into account EC technical guidance from 2006, and taking steps towards closer cooperation between cities and their surrounding regions. Such proposals are in line with the EC‟s Thematic Strategy on the Urban Environment, which is described above.

3.4.6 Energy Efficiency Plan (2011)

The Energy Efficiency Plan aims to secure smart, sustainable and inclusive growth, as a contribution to the development of a resource efficient economy, and the enhancement of the


03/01/2012 25 TTR

security of energy supply. Transport is identified as the sector having the second-largest potential for contributing to energy saving, (after buildings), and reference is made to the (then) forthcoming White Paper on Transport.

3.4.7 Ambient Air Quality Directive (2008 - On-going)

This Directive relates to ambient air quality and cleaner air in Europe‟s agglomerations, and has a focus on assessing, and then preventing or reducing, the harmful effects of poor air quality on human health and the environment. The planning document relevant to this Directive is the Air Quality Plan (AQP), which is analogous to the SUMP. The AQP is required should air quality levels fall below a defined threshold. The Directive specifies assessment regimes that relate to the measurement of specific emissions, such as NOx and PM10, at the level of individual agglomerations and urban zones.

3.4.8 Energy End-use Efficiency and Energy Services Directive (2006 - On-going)

This Directive relates to the commitment to control demand for energy sources through improvements to the efficiency of energy use by the end-user. The main objective is to contribute to reducing primary energy consumption, in order to mitigate CO2, and other greenhouse gas, emissions, so contributing to meeting Kyoto climate change commitments. The main focus of the Directive is the use of energy in the home, but it is also stated that the transport sector has an important role to play in energy efficiency and energy savings.

3.4.9 EC Directive on the Framework for the Deployment of Intelligent Transport Systems (ITS) in the field of Road Transport and for Interfaces with Other Modes of Transport (2010 - 2017)

This Directive was adopted in 2010, to accelerate the deployment of ITS throughout Europe. It is supported by the EC Action Plan for Deployment of ITS in Europe, which is described below.

3.4.10 EC Action Plan for the Deployment of ITS in Europe (2008 - 2014)

This Action Plan outlines six priority areas for action, including “European ITS co-operation and coordination”, and “The continuity of traffic and freight management ITS services on European transport corridors, and in conurbations”. Under the priority area of the continuity of traffic and freight management ITS services, it is proposed to define an ITS framework architecture for urban transport mobility. On the priority relating to European ITS co-operation and coordination, it was proposed to set up an ITS Collaboration Platform, from the Summer of 2010, to promote ITS initiatives in the area of urban mobility.

3.4.11 Promotion of Clean and Energy Efficient Road Transport Vehicles (2009 – On-going)

This Directive requires public authorities, and some operators, when procuring road transport vehicles, to take account of the lifetime energy and environmental impacts of these vehicles. Such considerations should include the energy consumption and emissions characteristics of the vehicles. It is intended that contracting authorities should, by their actions, promote and stimulate the market for clean and energy efficient public transport vehicles.

3.4.12 The Greening Transport Strategy Package (2008 – On-going)

This overarching package of strategies seeks to improve the sustainability of the transport sector in Europe. It seeks to do this, partly, through encouraging the price of transport to the consumer to more accurately reflect the real cost of transport to society as a whole, by considering environmental impacts and congestion. There are five parts to the strategy package: -


03/01/2012 26 TTR

Greening Transport Communication – which summarised the whole package, and described new initiatives that the European Commission was to introduce from 2009 onwards.

Greening Transport Inventory – which described EU actions currently in place for greening transport.

Strategy to Internalise the External Costs of Transport – which described how transport prices could be made to more accurately reflect the true cost to society of transport.

Proposal for a Directive on Road Tolls for Lorries.

Rail Transport and Interoperability Communication – which addressed the issue of noise levels associated with rail freight trains.


03/01/2012 27 TTR

4 Review of approaches to evaluation

4.1 Summary of findings on approaches to evaluation

An important activity was to conduct a review of current and recent projects and initiatives, in order to investigate the availability of evaluation and assessment frameworks which can help to improve the effectiveness of the sustainable urban mobility planning process. This was to further investigate the resources that are available for facilitating the implementation of EU policies at a local level. The remit for this activity was to examine what is currently being done in Europe in terms of urban mobility planning at a local level. Different approaches to evaluation and assessment have been compared, in order to ascertain the implementation and evaluation strategies that work well, and those that work less well. This was to further inform the development of the QUEST tool. The most relevant individual projects and initiatives are described in sub-section 4.3; details of those deemed to be of less relevance are contained in Appendix 3.

It was logical, however, to begin a state of the art review of a subject area by first considering the findings of similar reviews that have been carried out elsewhere. From the outset of the QUEST Project, the project team was aware of such a review being undertaken by the ELTIS+ Project. The results of this work were made available in July 2011, in the report “The State of the Art of Sustainable Urban Mobility Plans in Europe”. Our remit was to focus on information contained in this report on the subject of the evaluation of SUMPs and similar planning documents, of different approaches to carrying out such evaluation, and the the advantages and disadvantages of these different approaches. A summary of this information appears in sub-section 4.2.1, below.

There are two other key reports in this subject area. One of these is a report by an Expert Working Group on Sustainable Urban Transport Plans, commissioned by DG Environment in December 2004. This was linked to the Thematic Strategy on the Urban Environment. The other is the report “Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions“. This was published by DG Environment (Unit D4 - Health and Urban Areas), in October 2005. These reports are discussed in sub-sections 4.2.2 and 4.2.3, respectively.

The ELTIS+ State of the Art Review serves as a reference manual for urban mobility planning professionals, and has an emphasis on their training needs. It draws a number of conclusions on what constitutes a good SUMP. The ELTIS+ definition of the key characteristics of a good SUMP can be used as an authoritative initial evaluation check-list. It states that the essential objective of a SUMP is to ensure improvement on the three main themes of environmental sustainability, social cohesion and economic efficiency. It further disaggregates these themes to the important goals of ensuring access for all citizens, improving safety and security, and addressing specific environmental issues. There is an emphasis on stakeholder participation, the taking of an integrated approach to planning, and the setting of targets that are both measurable and achievable. The precise definition of objectives and targets, and the monitoring and evaluation of performance against these, are identified as being the key elements of a cyclical planning process. The SUMP planning process is described in the form of an eleven-step procedure, which is further broken down into 32 specific tasks.

The remit of the Expert Working Group on Sustainable Urban Transport Plans was the design of procedures, instruments & measures and advice on the content of any future EC Directive on SUMPs. The Working Group categorises its recommendations according to whether they should be obligatory or advisory. Among the obligatory recommendations is the involvement of stakeholders at every stage of the planning process, including that of


03/01/2012 28 TTR

monitoring and evaluation. Another is that evaluation should be carried out using a precisely defined set of indicators, which are set at the beginning of the planning process. The Working Group goes further, by recommending that the European Commission should define a compulsory set of indicators.

Among the individual projects and initiatives reviewed in some detail were those which have contributed to laying the foundations for the evaluation of sustainable mobility plans. These included PILOT, the CIVITAS Evaluation Programme and EPOMM Plus.

The PILOT Project had a strong emphasis on SUMPs. It demonstrated SUMP preparation in four European cities, and developed tools and guidelines, (such as the PILOT Evaluation Manual). The project highlights the importance of monitoring and evaluation, and also of Cost Benefit Analysis, within the planning process. In particular, there is an emphasis on the importance of carrying out evaluation using a monitoring and assessment framework based on common indicators. PILOT makes a contribution to improving the quality of the SUMP process by highlighting and interpreting lessons learnt from the demonstrations; it does this through the involvement of the project‟s Expert Working Group. It is the PILOT Project which provided the foundations for the ELTIS+ Project‟s recommendations for the minimum requirements for a good SUMP.

The CIVITAS Evaluation Programme is another important reference. CIVITAS is an initiative that has hosted many projects, including METEOR and GUARD, which coordinated evaluation activities in CIVITAS I and CIVITAS II, respectively. These projects covered both process evaluation and impact evaluation, and also issues concerning the transferability of measures. Whilst the full results of evaluations carried out for CIVITAS Plus are not yet available, this section contains an outline of the early findings of the CIVITAS Plus project ARCHIMEDES, a large-scale Demonstration Project. The main principles emerging from ARCHIMEDES are the importance of identifying stakeholders, and then engaging with them at every stage of the planning process, the definition of precise targets and indicators, and the provision of evaluation reports on an annual basis. There have also been specific lessons learnt about procedures for stakeholder participation.

EPOMM Plus is another current initiative that has been reviewed in some detail. This project focuses on the implementation of Mobility Management in Europe, and the dissemination of good practice. It also develops training initiatives, and uses a suite of evaluation and decision support tools previously developed by the Max Project. These tools include MaxExplorer (decision support), MaxEva (evaluation), MaxSumo (standardising evaluation), MaxQ (Quality Management) and MaxSem (behavioural and psychological modelling).

Other projects summarised are those dealing with: -

decision making and participation strategies in planning (GUIDEMAPS)

support for implementing and monitoring sustainable mobility plans (BUSTRIP)

the removal of barriers to delivering sustainable urban mobility (DISTILLATE)

the establishment of a tool-kit of methods and standards that can be used to change attitudes and behaviour (MOVE)

the demonstration of sustainable transport means (CATCH, CO2NeuTrAlp)

guidance on optimal land-use and transport strategies (PROSPECTS).

4.2 Findings of previous state of the art reviews and reports by the ELTIS+ project and DG Environment

4.2.1 State of the art review and guidelines from the ELTIS+ project

ELTIS (European Local Transport Information Service) is an initiative funded by Intelligent Energy Europe, and managed by EACI (the Executive Agency for Competitiveness and


03/01/2012 29 TTR

Innovation), and has the aim of encouraging the exchange of information and experiences in the field of urban transport and mobility. As Europe‟s main portal on urban mobility, it was relaunched in March 2011 with a new web-site providing information about the benefits of SUMPs and also guidelines on the development and implementation of SUMPs. The State of the Art document published in July 2011 as a deliverable of the ELTIS+ Project serves as a reference manual for urban mobility professionals. There is some emphasis on training needs in relation to SUMPs, but the report also provides a review of sustainable urban mobility planning in all 27 EU Member States, plus Norway, Liechtenstein, Croatia and Iceland, and there is a discussion on what consititutes a good SUMP.

The ELTIS+ project describes the creation of SUMPs as a “new way of planning”, and sets the scene for the evaluation of such plans by presenting a comprehensive definition of a SUMP, which is in turn based upon the definition developed by the PILOT project in 20073. Such a definition might be used as an initial evaluation check-list, in as much as it sets out the primary aims and objectives against which a SUMP should be assessed; these are to: -

ensure the transport system is accessible to all

improve safety and security

reduce air and noise pollution, greenhouse gas emissions and energy consumption

improve the efficiency and cost-effectiveness of the transportation of persons and goods

contribute to enhancing the attractiveness and quality of the urban environment and urban design4.

The detailed definition goes on to state that a SUMP should entail a participatory approach which involves citizens and stakeholders at every stage of the planning process, including the evaluation stage, and that there should be a focus on balancing economic, environmental and social equity priorities. A key characteristic of a SUMP is that it should represent an integrated approach which brings together different policy sectors (e.g. transport, land-use, environment, social inclusion, safety, etc.) and different authority levels (i.e. local, municipal, regional, etc.), and also coordinates policies and practices across geographical administrative boundaries. Whilst a SUMP is embedded within an overall strategy for sustainable development, it focuses on measurable, achievable targets. Generally, a SUMP should comprise five key stages: -

1. Status analysis and baseline scenario

2. Definition of objectives and targets

3. Selection of policies and measures

4. Assignment of responsibilities and resources

5. Monitoring and evaluation

It is stages “2.“ and “5.“ which are of primary interest to the current QUEST deliverable.

The ELTIS+ document also puts forward a suggested short definition of a SUMP: -

“A strategic plan designed to satisfy the mobility needs of people and businesses in cities and their surroundings for a better quality of life. It builds on existing planning practices and takes due consideration of integration, participation and evaluation principles”.5

3 PILOT Project. Sustainable Urban Transport Plans – SUTP Manual, Guidance for Stakeholders. (2007).

4 ELTIS+. State of the Art of Sustainable Urban Mobility Plans in Europe. (2011).

5 ELTIS+ Project (2011) (op cit).


03/01/2012 30 TTR

The document goes on to elaborate on the characteristics that a plan should have, in order to be considered to be a SUMP, rather than a traditional transport plan. These include having a long-term strategic vision over a 20- to 30-year time horizon, evidence of cooperation with authorities beyond the bounds of the main city, a high level of citizen and stakeholder involvement, a decisive shift of emphasis towards more sustainable forms of transport, and evidence of the cross-sector and multi-level integration described above. In terms of evaluation, the essential characteristic of SUMPs is the focus on the achievement of measurable targets, outcomes and impacts, and then evidence of a cyclical repetition of the planning process, in order to foster continuous self-improvement.

The role of evaluation and monitoring are briefly described within the context of an eleven-step procedure for creating and implementing a SUMP, which is presented both in the ELTIS+ State of the Art document, and in a document that fulfils the role as an annex6. These eleven steps are as follows: -

1. Determine your potential for a successful SUMP

2. Define the development process and scope of plan

3. Analyse the mobility situation, and develop scenarios

4. Develop a common vision

5. Set priorities and measurable targets

6. Develop effective packages of measures

7. Agree on clear responsibilities and allocate funding

8. Build monitoring and assessment into the plan

9. Adopt sustainable urban mobility plan

10. Ensure proper management and communication

11. Learn the lessons7

These stages are further broken down into a total of 32 specific tasks and activities. Tasks that specifically relate to evaluation are “5.2 Develop SMART (i.e. Specific, Measurable, Achievable, Realistic, Time-bound) targets”, “8.1 Arrange for monitoring and evaluation”, “10.3 Check progress towards achieving the objectives” and “11.2 Review achievements – understand success and failure”.

The guidelines which accompany the State of the Art document provide further detail as to the rationale, aims and detailed procedures for the monitoring and evaluation of SUMPs.

The major points made on this issue are: -

ideally, SUMP evaluation should be carried out by a body that is otherwise independent of the process, and which has the necessary skills for carrying out the required work

responsibilities for carrying out the evaluation work should be clearly defined, and there should be a set work plan

a mechanism for monitoring and evaluation should be defined early on, so that it can become an integral part of the planning process; this enables drivers and potential hitches in the SUMP process to be anticipated. Ideally, extensive involvement of stakeholders should be built into all stages of the process, and consideraton should be given to having peer reviews from similar cities

6 ELTIS+ Project. Guidelines: Developing and Implementing a Sustainable urban Mobility Plan. (July 2011).

7 ELTIS+ Project (2011) (op cit).


03/01/2012 31 TTR

criteria for whether targets are met, and measures are implemented, should be set in advance

selection of indicators for evaluation should be linked to the earlier setting of SMART targets; both quantitative and qualitative indicators should be used

an audit should be carried out into the availability of data, and into what gaps in the data might exist

the evaluation mechanism should address both the quality of the planning process, and the outputs (actions taken, such as the provision of an additional x km of cycle lanes) and outcomes (the impacts of these actions, such as evidence of improved quality of life, reduced congestion, etc.)

the budget set aside for monitoring and evaluation activities should be at least 5% of the total budget for the SUMP

the results of the evaluation should not just be reported within the circle of practitioners involved in the process, but also be disseminated to the public domain, in order to inform the public debate; all stakeholders should have the opportunity, at this stage, to suggest amendments to the SUMP.

The guidelines state that a cost-benefit analysis of the SUMP process and its impacts might be carried out, but do not consider this to be essential.

The document provides actual examples of the monitoring of sustainable mobility plans; these examples are from the Local Transport Plan (LTP2) for West Yorkshire, in the UK, and the Plan de Déplacements Urbains (PDU) for Toulouse, France.

A list of specific indicators used for evaluating the West Yorkshire LTP is quoted, along with details of the data source for each target, the collection techniques to be used and the anticipated timescale involved. Both qualitative and quantitative examples are given: the indicator “Bus Punctuality“ was monitored by means of roadside surveys and the Real Time Passenger Information system, with updates anticipated annually; more qualitative information on satisfaction with local bus services was obtained through a combination of Central Government survey statistics, and market research surveys carried out by the local Public Transport Executive, with these data being updated once every three years.

The French example provides details of a number of monitoring mechanisms that are woven into the planning process, some of which are mandatory requirements under French planning law. Involvement of both public and private stakeholders has been ensured with the setting up of a “partnership“ monitoring commission comprising a number of institutions, associations and mobility-related organisations, which meets at least once a year. The activities of this commission are supplemented by the statutory provisions that are in place in France, such as the requirement for there to be a full evaluation of the PDU every five years. This requirement is laid down by the Loi sur l‟Air et l‟Utilisation rationnelle de l‟Énergie (LAURE), the law covering air quality and energy consumption, which dictates that all urban areas of more that 100,000 population must provide a PDU. The evaluation is informed by an “observatoire”, which monitors the actions planned in the approved PDU document, the development of indicators used for evaluation, and the direct and indirect impacts of actions taken. The observatoire serves as a tool for managing the decision-making process, and as a means of communication with stakeholders. It poses direct questions, such as “Have planned actions been effected ?”, and “Are the principles of the PDU being upheld ?”, and “Are the impacts of actions as anticipated ?”. As well as this five-year strategic evaluation, an annual interim evaluation is carried out. Other monitoring data available to the Toulouse authorities is available from a household mobility study, a study of the city‟s ring road and a public transport origin-destination study. The evaluation and development process is also supported by an Urban Development / Mobility Commission, which enables collaboration between the authority charged with responsibility for the SUMP, and the authority responsible


03/01/2012 32 TTR

for Toulouse‟s urban development coherence scheme. The French Government also provides two other tools – whose use is obligatory – to facilitate the evaluation process. One of these is a mobility cost accounting method, which measures costs to both users of a scheme and to society as a whole; the other is a balanced scorecard method, which is a strategic performance management tool, which can be used to track actions completed, and their consequences.

One of the recommendations made in the description of Task 10.3 “Check progress towards achieving the objectives” is that monitoring and evaluation activities should lead to a “repackaging” of measures, in order to ensure that targets continue to be met in the most efficient way possible. It is also re-emphasised that the results of evaluation should be shared in the public domain, in order to provide stakeholders with the opportunity to engage in debate and suggest changes to the plan, where appropriate.

The guidelines are again illustrated with actual examples of monitoring regimes. One example is drawn from Aachen, Germany, where a stakeholder group consisting of Aachen city‟s transport and environment departments, the chamber of industry and commerce, the local public transport operator and the regional transport association meets on a quarterly basis. A second example is provided by the city of Vitoria-Gasteiz, which monitors its sustainable mobility plan (“Plan de Movilidad y Espacio Público”) by means of a survey. The first part of this survey consists of 4,000 telephone interviews to evaluate the plan; the second part consists of interviews with 300 companies and 2,700 employees, and focuses on mobility in relation to economic activities within the city. The most recent such survey is scheduled for 2011, the previous one having been carried out in 2006.

4.2.2 Report of the Expert Working Group on Sustainable Urban Transport Plans (December 2004)

The Expert Working Group on SUTPs, (which, for consistency, will be referred to as SUMPs in this section), was set up to support preparation for the Thematic Strategy on the Urban Environment. This document reports on the findings of this Expert Group. The Group‟s remit was to take a view regarding the specific design of procedures, instruments and measures needed for SUMPs, the contents of a potential EC directive, and the support actions necessary for enhancing SUMPs. Recommendations for devising an EC Directive are weighted according to whether they constitute obligations or advice. It is the obligatory recommendations that constitute the minimum requirements for a SUMP.

The Expert Group defines a five-step implementation mechanism for SUMPs, which includes “Monitoring and Evaluation” as a fifth step. One obligatory recommendation is that engagement with stakeholders must be built into every stage of the process, including that of evaluation, and that these stakeholders must be representative of all sections of society. (Gender equity and equality is given particular emphasis). Also obligatory is the need for the SUMP implementation to be monitored according to a set of indicators defined early on in the implementation process. It is suggested that regular progress reports should be prepared at regular intervals (e.g. annually), and that these reports should be published widely. The Group specifies that the evaluation process should consider both the quality of the SUMP, and the quality of the implementation process. It is also recommended that periodic “sanity checks” should be carried out, with the input of stakeholders, members of the public and peers from other cities.

The document provides a detailed description of what the EC should include as being obligatory and advisory in a directive that is designed to encourage the widespread creation and implementation of SUMPs. A detailed recommendation is that the EC should define a compulsory set of key indicators to be used, and that these indicators should be rigorously defined, in order to ensure comparability. Reference is made to several large sets of indicators already available, as an example of the indicators that might be used. It is also recommended that the EC should define minimum targets for these indicators, based on


03/01/2012 33 TTR

existing EC environment and sustainability legislation – these defined minima should form the basis of targets set at a national level, which should be set by Member States, who should also be responsible for monitoring performance against these.

The report identifies France, Italy and the UK as being the most advanced European countries in terms of national regulations for local transport planning, and so makes a comparative analysis of the situation in the three countries.

4.2.3 Report: Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions (October 2005)

The report was commissioned in recognition of the contribution that the transport sector makes to emissions of NOx, PM, VOC, O3, CO2 and noise in cities. The document provides a review of current empirical evidence for the impact of SUMPs on health conditions in cities, and overall quality of life, and identifies current knowledge gaps. There is an evaluation of SUMPs from the point of view of their impact on NOx, PM, VOC, O3, CO2 and noise, according to available evidence. In practice, the report covers over 50 specific assessments of policies which target transport emission reduction. The four broad categories of policy featured are policies which reduce the need for motorised transport, encourage modal shift to more sustainable means, develop clean and quiet means of transport and improve transport efficiency. A major aim of the study is to inform policy makers as to the extent to which there are complementarities between the various studies. The study also specifically considered different policy modelling approaches that were available. The approach of the study was to prioritise assessments which provide a quantitative assessment of policy impacts (on emissions and noise, etc.), and qualitative advice on the creation of integrated transport policy packages to maximise impacts and thus sustainability.

Policies and measures are categorised and dealt with in four categories, according to their main emphasis: 1. Reduction of the need for motorised transport; 2. Encouragement of modal shift; 3. Development of cleaner, quieter transport systems; 4. Improvements to transport efficiency. It is suggested that these are the fundamental policy categories that should be used as a basis for policy design. The study finds that there are numerous synergies between the policies that make up SUMPs, and also identifies some adverse impacts. These synergies and impacts are examined in detail, in what the report describes as an analysis of “SUMP efficiency”.

4.3 Approaches to evaluation: Current European projects and initiatives

A number of projects and initiatives funded by the European Commission have been reviewed for this work package. The most relevant of these are described, in turn, in the following sub-sections, with a summary of each project / initiative appearing in Appendix 3.

4.3.1 PILOT (Planning Integrated Local Transport)

PILOT, a DG Environment Project that ran from 2005 to 2007, had a strong focus on Sustainable Urban Mobility Plans. The aim of the project was to demonstrate the preparation of SUMPs in four European cities: Braila, Evora, Lancaster and Tallinn. Building upon the experiences of these four cities, and using the knowledge of experts from leading local authorities and organisations in this field, PILOT proposed tools, guidelines and recommendations for the creation of SUMPs in other European regions and local authorities. Specifically, a guidance document was produced which provided a comprehensive approach to defining, setting up and running the SUMP process. In this way, the PILOT Project made an important contribution to encouraging the implementation of European-wide policies for the creation of a more sustainable transport system, at a local and regional level. The project emphasises the fact that the production of a detailed monitoring and evaluation plan makes it possible to anticipate potential difficulties and risks in the creation and implementation of a


03/01/2012 34 TTR

SUMP, and that, in much the same way, post-implementation evaluation provides the opportunity for the strengths and weaknesses of the planning process to be assessed. This means that improvements might be made for subsequent planning and implementation. The evaluation of the costs and benefits (i.e. the impacts) of the SUMP process also constitutes a justification of the sustainable planning process, and one that is valid at a local, national and European level. Featuring examples of cities‟ experiences with the creation and implementation of SUMPs, with the input of experts for interpreting the lessons to be learnt, was another means by which PILOT contributed to maximising the effectiveness of the implementation of these policies.

An early activity of the project was the establishment of an Expert Working Group (EWG), whose aim was to provide balanced stakeholder feedback on what a potential EC Directive to provide a framework for SUMP regulation in all Member States should contain. The Group made a number of recommendations, including a definition of what the minimum requirements for SUMPs should be. This definition included: -

the establishment of a common evaluation framework

the provision of financial support

guidance

dissemination of best practice, with associated training

co-ordination of current policies.

The EWG went further, in considering the component parts that were required for SUMPs to be used as an effective planning tool. This has become accepted as a framework for defining good practice. Its main elements are: -

citizen participation

stakeholder consultation

actor co-operation

policy co-ordination

integration, with comprehensive planning

land use planning.

An important objective of PILOT was the development of a common indicator-based monitoring and assessment framework for SUMPs, and this became a key recommendation to emerge from this project.

Another important recommendation, contained in the project‟s final guidance document, was that, in order to enable local policy learning, both the SUMP process and the implementation of the action and budget plan need to be assessed by an independent body. This implies a monitoring of actions at each step of the planning process, as well as measuring eventual policy outcomes and impacts. The results of the evaluation should then feed back into the process on a regular basis, either to improve the planning process, or to review the measures adopted.

Other criteria specified when evaluating the success of a SUMP overall were: -

a well-balanced selection of indicators

evidence of clear responsibilities defined for well-skilled staff members for monitoring and evaluation activities

the existence of a clearly defined budget and workplan for monitoring and evaluation

a focus on ex-post evaluation on outcome indicators

a clear assessment of the costs and benefits of SUMPs

stakeholder involvement in monitoring and evaluation


03/01/2012 35 TTR

The project also emphasises the importance of involving stakeholders in monitoring and evaluation activities, and encourages seeking feedback from peers from other cities, and the co-ordination of regional indicators.

The PILOT Evaluation Manual also sets out a sequence of milestones that should be achieved: -

1. The finalised selection of suitable indicators

2. Agreement on a work plan and responsibilities for data collection and management

3. Ex-ante evaluation finalised (scenario development, action and budget plan)

4. Ex-post evaluation of plan implementation and impact finalised

5. Ex-post evaluation of plan preparation finalised

6. Finalised evaluation reports

All of these stages of development and activities are summarised in a flow diagram that is published in the PILOT Evaluation Manual; this shows the inter-relationships between all stages of the monitoring and evaluation process, from the selection of indicators and scenario development, to the ex-post evaluations8. The manual also confirms that the evaluation process should cover both process evaluation (perhaps involving participatory observation) and impact evaluation (entailing some measurement and survey work), and that both quantitative and qualitative indicators should be used.

4.3.2 The CIVITAS evaluation programme

CIVITAS is a broad initiative that was set up to help cities to achieve a more sustainable, clean and energy efficient urban transport system, by implementing and evaluating an ambitious, integrated set of technology and policy based measures. To date, there have been three rounds of CIVITAS projects, I, II and Plus. Within CIVITAS I (2002-6) there were 4 projects with a total of 19 cities involved. Within CIVITAS II (2005-9) there were 4 projects with 17 cities involved. Within CIVITAS Plus (2008-2012) there are 5 projects, with 25 cities involved. Bids to a new round of Plus II projects are currently being considered. What is of interest to the QUEST Project is that each phase of CIVITAS has involved a cross-project evaluation. The CIVITAS I evaluation activity was co-ordinated by the METEOR consortium. The CIVITAS II evaluation activity was co-ordinated by GUARD. (CIVITAS Plus projects are ongoing, so evaluation results are not yet available.)

Within the cross-project evaluation programmes, many different types of sustainable transport measures were evaluated, including SUMPs. The key elements of evaluation included: -

process evaluation to understand the process behind measure development and implementation - this identified barriers and drivers, and assessed the level of influence of these on development and implementation

impact evaluation, including technical, social and economic impacts, using various indicators. This involved comparing situations before and after measure implementation, and comparing the impacts identified with the results of a „do-nothing‟ scenario. Impact evaluation also involved cost-benefit analysis, based on a standard formula, but this was conducted only for a limited number of measures, due to issues with the quality of data.

assessment of the transferability of measures to other locations.

8 SUTP Manual: Guidance for Stakeholders. Task 5: Monitoring and Evaluation. (PILOT Project, August, 2007).


03/01/2012 36 TTR

The following lessons were identified from various GUARD (CIVITAS II) evaluation reports. While these were not specific to SUMP evaluation, they are nevertheless pertinent to the issue of evaluation: -

maintaining a proper balance between delivery and evaluation, in terms of resourcing, is a difficult issue which needs regular attention. Resource constraints can lead to inappropriate evaluation approaches being used. From the start of a measure, up to 10% of the budget may be needed for evaluation. An evaluation plan is required

the selection of indicators for evaluation should be based on the following criteria: -

- relevance

- completeness

- availability

- measurability

- reliability

- familiarity

- independence

too often, a focus was placed on a limited set of indicators

on-going evaluation is necessary, from the very beginning of the process of measure development. Evaluation should start with an analysis of the current situation, including the collection of existing data. Subsequent evaluation should show the lasting effects of an intervention, and suggest the frequency at which an intervention should be reviewed and, if necessary, adjusted

some measures might develop over time, meaning that evaluation requirements might also change. Similarly, delays in implementation might require changes in evaluation to be made. This might create issues that are difficult to overcome, particularly if some “before” data have already been collected. This requires careful management.

local politicians may not welcome publication of results which show “failure” of a measure.

Some of the lessons from CIVITAS Evaluation Programme are similar to those revealed by the current CIVITAS Plus ARCHIMEDES project, which is discussed in the following sub-section.

4.3.3 ARCHIMEDES (Achieving Real Change with Innovative Transport Measures Demonstrating Energy Savings)

ARCHIMEDES is a large-scale demonstration project, funded as part of the CIVITAS Plus Initiative (2008-12). It addresses problems with, and opportunities for, creating environmentally sustainable, safe and energy-efficient transport systems in medium-sized urban areas. These areas are categorised into “Leading Cities” which transfer experience and best practice, and “Learning Cities” which benefit from this dissemination. Project measures include the development of SUMPs by the two Learning Cities, Monza (Italy) and Ústí nad Labem (Czech Republic). Leading Cities make their expertise available to the Learning Cities through training and learning activities, including training workshops for stakeholders.

Whilst detailed evaluation procedures for Learning Cities will not be finalised until towards the end of the project, (which is September 2012), approaches to SUMP evaluation in ARCHIMEDES, as identified in training activities, include: -

self assessment


03/01/2012 37 TTR

identification of all relevant stakeholders (including citizens), and involving them in all stages of the SUMP process, including evaluation

the definition and monitoring of targets and indicators

process evaluation, which is a continuous activity from the outset

impact evaluation, which occurs after the implementation phase, in order to verify achievements and policy outcomes

evaluation of SUMP planning, implementation and budget. This includes establishing procedures to handle discussion of detailed actions

the production of annual progress reports.

One of the general lessons learnt during ARCHIMEDES is that targets should be measurable, quantifiable, relevant and realistic, in order to allow effective monitoring of the SUMP„s progress, and assessment of the efficiency and effectiveness of the measures taken. Work on the project has proceeded on the basis of a SUMP not being a closed planning tool, but a cycle of policy making, planning and implementation. It has been important, therefore, that the evaluation mechanisms adopted should allow assessment and modification of efforts over time, and inform future improvements. This is a central principle of Total Quality Management).

Some lessons have also been learnt specifically in relation to stakeholder participation; these are: -

stakeholder representation should be representative and inclusive, but also manageable and cost-effective

the interests, resources and capacities of different stakeholders should be taken into account

the planning of the main involvement activities should be concluded before the SUMP process begins

different engagement tools and techniques should be considered, and the most suitable selected, according to the local context and/or target group

stakeholders should be provided with sufficient and transparent information regarding the SUMP in question, to ensure informed involvement and confidence in the process

dealing with changes in stakeholders„ role can be challenging, (in terms of continuity of input, for example).

4.3.4 EPOMM PLUS (European Platform on Mobility Management PLUS), and the Max Project suite of tools

EPOMM PLUS is a current initiative (2009-12) that is funded under the IEE STEER Programme. It is a network of governments in eleven European countries - Belgium, Bulgaria, Czech Republic, Estonia, Greece, Hungary, Italy, Lithuania, Poland, Romania, SIovenia – which focuses on the issue of Mobility Management (MM). As its title suggests, EPOMM PLUS builds upon the earlier EPOMM initiative, and has an emphasis on the newer EU Member States and the countries with less MM experience. The overall objective of the EPOMM PLUS project is to achieve a leap in the implementation of MM in Europe, by establishing EPOMM as the authority and the networking instrument for the promotion of MM across the EU. This is a very important tool that the European Commission can use for ensuring that sustainable mobility management is integrated into national transport policies.

This is achieved at a national level by enabling the dissemination of good practice, and developing national networks to provide support for local pioneers. The building of national networks and platforms to facilitate such dissemination is supported through EPOMM PLUS partners, (known as Network Initiators), in 22 European states. The dissemination of best practice is also facilitated through the work of Mobility Management Monitors, who enable


03/01/2012 38 TTR

mobility management road maps to be exchanged between EPOMM countries. National Event promotion also encourages such dissemination; included in this work is the translation of EPOMM materials into the local language of the event, as well as the continual updating of the EPOMM PLUS web-site, the promotion of National MM Workshops, and the distribution of electronic updates.

EPOMM PLUS also develops training initiatives, which includes “train the trainer” meetings. Included in the project‟s work is the provision of a training quality label, along with a training and trainer database.

The monitoring and evaluation of mobility management projects is carried out using tools, such as MaxExplorer and MaxEva, developed by the Max Project (“Successful Travel Awareness Campaigns & Mobility Management Strategies”) (2009), which was co-funded by the European Commission within the Sixth Framework Programme for Research and Technological Development. MaxExplorer is an interactive decision support guide for mobility management projects. It can assist decision makers and mobility management practitioners to identify measures rated by Max Project experts as being the most appropriate for specific situations. The Max project developed a quality management approach as a powerful tool to improve mobility management in general, but especially for cities. It helps to provide services in an organised and consistent manner and to continuously improve them based on user satisfaction and desires.

The Quality Management System () developed by the Max Project focuses on developing, monitoring, assessing and improving both the overall MM policy, and individual MM measures. is a process, which involves four steps: Policy, Strategy, Implementation and Monitoring & Evaluation. These steps are comprised of twelve sub-steps, which are structured in a “quality circle”. The use of this Quality Management tool can be beneficial in: -

introducing the technique of Quality Management to mobility operations

achieving quality while adopting a “quality culture“ to the provision of services

improving the performance and credibility of mobility management processes.

Planning, monitoring and evaluation in mobility management can all be facilitated by the use of two other products of the Max Project: MaxSUMO and MaxEva.

MaxSUMO aims to fulfill the important task of standardising evaluation, in relation to mobility management projects at a European level. It provides standardised guidance for all steps of an MM project, including the setting of targets, the definition of target groups, and the selection of services and mobility options. Use of a standardised procedure for the evaluation of mobility planning and management enables more effective gauging of the extent to which policies and legislation created at a European level are implemented in a local, regional and national context. A description of the use of this evaluation tool, in the context of the MOVE Project, is provided in sub-section 4.3.11.

MaxEva provides an on-line database of cities‟ evaluation data in relation to mobility management projects. Its objective is to gradually grow this database in order to provide a valuable source of knowledge for the planning of new projects. MaxEva is an interactive web tool that provides guidance for the collection of data for monitoring and evaluating mobility management interventions. The tool is designed to provide a continuously updated overview of results, in relation to the project that is being analysed, based on data that have been entered. If the necessary data are made available to the tool, it can calculate specific impacts, such as reduced mileage travelled, reduced levels of CO2 emissions, etc.

MaxSem is an additional tool of the Max pproject which can assist the monitoring and evaluation of mobility management projects. It is a psychological model which explains the underlying processes involved in changing the behaviour of car drivers, in the context of their propensity to switch to using more sustainable means of transport. MaxSem provides the means to gain a “fuller picture” of the effects of MM, and assists in the segmentation of target


03/01/2012 39 TTR

groups. Furthermore, it is helpful to understand the underlying processes that are necessary for behavioural change to occur, and to use this knowledge in both the design and evaluation of MM projects.

For practitioners, it is very important to be able to measure constructs such as behaviour change, in order to be able to provide evidence of the impact of policies and actions. The monitoring and evaluation of the beneficial impacts of policy interventions is clearly important to being able to secure political support and funding for further similar measures. EPOMM PLUS, through its use of the Max Project suite of tools, helps the user, in a variety of ways, to effectively plan, monitor and evaluate MM measures and projects. Changing people‟s behaviour is a long-term activity, and occurs in steps, or stages. Stage-diagnostic questions have been developed that can be used to show stage movements as one of the effects of a policy intervention, in addition to measuring overt behaviour change.

4.3.5 GUIDEMAPS (Gaining Understanding of Improved Decision Making And Participation Strategies)

GUIDEMAPS was a project co-funded by the European Commission within the Fifth Framework Programme for Research and Technological Development, from 2002 to 2004. The principle aim of the project was to identify procedures and tools to both improve policy decision making, and achieve sustainable mobility throughout the European Union, by overcoming barriers and delivering better policy outcomes. Detailed objectives of GUIDEMAPS were to: -

carry out a state of the art review of decision making and engagement in the field of transport planning in Europe

identify barriers to successful decision making

identify procedures and tools for improving policy decision making, and achieve sustainable mobility throughout the EU

propose a set of guidelines and tools for overcoming barriers, and for designing an efficient project management process.

The project included a broad overview of current practice in decision making and public participation processes. This was followed by an identification and analysis of barriers to the adoption of better management and decision-making techniques and processes in transport planning. The main output of the project was a set of guidelines, which were tested through a series of case studies, and then disseminated through training courses. These guidelines are contained in the handbook “Successful Transport Decision-making”.

The handbook gives a practical overview of good practice in stakeholder engagement, public participation and project management for local and regional transport projects. It is designed to support the decision makers involved in local and regional transport planning in Europe, and has an emphasis on three key concepts: Barriers, Project Management and Engagement. For example, advice on engagement techniques provides detail on the different communication tools that might be used (e.g. printed material, telephone and internet), techniques for interactive engagement (e.g. exhibitions, focus groups, stakeholder conferences), and ideas for engaging with people from more “hard to reach“ groups (e.g. disabled people, people with low levels of literacy, people from ethnic and religious minorities).

The handbook is primarily aimed at transport professionals working in local authorities or transport companies, but it is also relevant to all stakeholders involved in the decision making, engagement and project management process. These might include elected officials, community leaders, transport operators or financiers, campaign groups, NGOs and interested citizens. The relevance of the handbook to such a broad range of actors is important, since successful transport decision making and implementation depends upon


03/01/2012 40 TTR

both being accepted by a large number of people, and by the main stakeholders, in particular.

Whilst GUIDEMAPS project reports make no specific reference to the evaluation of SUMPs, the project has relevance to the issue of evaluation in as much as it provides an authoritative benchmark for good practice against which evaluation approaches can be measured. Such good practice is also contained in a series of Fact Sheets, which provides detailed and practical advice on different techniques that can be used, and on how they should be applied. One of the conclusions from the survey undertaken by GUIDEMAPS is that there was evidence of wide variation in the practice of decision making between and within European countries. This supports the argument for encouraging the harmonisation of the techniques that are used for the evaluation of mobility planning and management activities.

Subject areas covered by the GUIDEMAPS Fact Sheets are: -

measuring indicators

tools for tracking progress

measuring outcome indicators

post implementation evaluation.

A more in-depth description of the GUIDEMAPS project appears in Appendix 3.

4.3.6 BUSTRIP (Baltic Urban Sustainable Transport Implementation and Planning)

BUSTRIP was an INTERREG IIID project for the Baltic Sea region, which took place from 2005 to 2007. Its remit was to support partner cities in developing transport planning processes, preparing new urban sustainable transport plans and revising existing plans. Evaluation was carried out using a process of self assessment and peer review.

The peer review includes a gap analysis of the actual performance of the city, in comparison with the SUMP benchmark defined by the project. The peer review report follows a structured methodology and template, both prepared by the BUSTRIP Project, and designed to enable comparisons to be made. The results of the peer review intended to assist participating cities to prepare or improve their SUMP.

The approach of the project in helping cities to evaluate their own sustainable planning efforts was to first provide a simple “Yes / No” check-list, in order to focus participants‟ mind on the areas most in need of attention and effort. This check-list was divided into no fewer than 20 sections: -

1. Organisation of sustainable planning

2. Stakeholder participation

3. Coordination of policies and plans

4. Capacity building

5. Drivers (i.e. the factors which influence traffic levels etc.)

6. Analysis of current situation

7. Gender equity and equality

8. Scope and definition of plan

9. Analysis of baseline scenario

10. Vision, objectives and targets

11. Implementing action, allocating finance

12. Policies and measures

13. General principles

14. Reducing the need for transport

15. Transport management


03/01/2012 41 TTR

16. Clean and silent transport system

17. An accessible and safe transport system


19. Plan adoption, approval and assessment


By way of example, the following is a short list of some of the questions that were included: -

has a SUMP9 Working Group been established ?

has responsibility for SUMP work been defined ?

is there political support for SUMP work ?

have relevant stakeholders from within the municipal organisation been identified ?

have relevant regional and national stakeholders been identified ?

have policies and rules that affect transport been identified and listed ?

is there sufficient manpower to work with SUMP issues ?

is there sufficient know-how to work with SUMP issues ?

have drivers for transport development in the city been identified ?

have development trends for each driver been listed ?

are statistics and regular measurements on traffic development available ?

is there a balanced gender representation in the SUMP working group ?

is equal accessibility for all groups considered in planning and the decision making process ?

is the scope / goal of the SUMP clearly defined ?

does the city have a set of measurable indicators that can be used to describe the traffic situation and traffic impacts in the city ?

has a baseline year against which traffic development is compared been chosen ?

have target values been defined for all of the city‟s transport indicators ?

Under the “Monitoring and Evaluation” heading, there are six questions: -

is it defined how progress towards SUMP goals will be monitored ?

are there set dates for evaluation of progress of the SUMP work ?

is there a clearly defined unit / person responsible for evaluating the progress of SUMP work against the defined goals and targets ?

has that unit / person the necessary competence, tools and resources for the evaluation of progress towards SUMP targets ?

is there a time plan for evaluation of SUMP progress ?

is it defined to whom the evaluators will report their results ?

Further explanation of the questions, with example answers, is provided in Section 2 of the document.

A full list of the questions included in BUSTRIP‟s check-list appears in Appendix 6.

Lessons learnt from BUSTRIP, in relation to evaluation, were that: -

9 The term used in the BUSTRIP Project was Sustainable Urban Transport Plan (SUTP), but “SUMP” is used here, in the interests of maintaining a consistent terminology.


03/01/2012 42 TTR

a structured and well prepared self assessment greatly facilitates the review process; the BUSTRIP Project created a self assessment template for this purpose

peer reviews need thorough preparation, and need to be implemented by an organised review team. The questions for, and an explanation as to the purpose of, the peer review need to be clearly defined in advance, and the auditor needs to make a thorough desk review of the self assessment report before visiting a city

the assessment needs to take the local context and national legislation into consideration, since what cities are empowered to do varies between nations.

4.3.7 DISTILLATE (Design and Implementation Support Tools for Integrated Local Land Use, Transport and the Environment)

DISTILLATE (2004-8) was a focused, inter-disciplinary research programme, whose aim was to study ways of overcoming the barriers to effective development and delivery of sustainable urban transport and land use strategies and, through them, enhanced quality of life. It was funded by the UK‟s Engineering and Physical Sciences Research Council‟s overarching research programme on the development of a Sustainable Urban Environment, and was divided into seven projects. These covered a range of different activities involved in the development of sustainable urban transport and land use strategies. The projects were: -

Project A - Organisational Behaviour and Barriers

Project B - Improved Tools for Option Generation

Project C - Improved Indicators for Sustainable Transport & Planning

Project D - Improved Effectiveness in Organisational Delivery

Project E - Improved Mechanisms for Funding and Phasing of Implementation

Project F - Enhanced Analytical Decision Support Tools

Project G - Enhanced Appraisal Tools.

Whilst none of these projects directly addressed SUMP evaluation per se, the initiative is relevant to QUEST in as much as it used a number of different tools to assist in evaluating and supporting decision making processes. The work had a strong focus on performance management and the use of indicators, which are central to the process of evaluation, and there was some detailed examination of measures of sustainability and related indicators.

Project C, “Improved Indicators for Sustainable Transport & Planning“, was the most relevant to the work of the QUEST Project. Part of the research brief for this project was to: -

conduct a survey of local authorities‟ experience in measuring, predicting and using indicators

determine the extent to which current indicators correspond to stakeholders‟ understanding of sustainability and quality of life

specify the requirements for a core set of indicators at each stage in the decision-making process

Identify a core set of outcome indicators that best meets those requirements.

The survey work for Project C identified a set of concerns surrounding the ways in which indicators are applied in practice in the UK. Eight aspects of indicators were allocated importance levels between “fairly important“ and “very important“, and levels of satisfaction between “not satisfied“ and “fairly satisfied“. These aspects of indicator selection would therefore appear to be of greatest concern to the practitioners. In order of importance, these were: -

1. The use of indicators in the development of well-founded targets

2. The cost effectiveness of monitoring


03/01/2012 43 TTR

3. The ability to capture year-on-year improvements

4. Ease of measurement

5. Ease of understanding of the process by politicians

6. Ease of understanding of the process by the general public

7. Poor Consistency between transport and planning indicators

8. Poor Consistency between transport and sustainability indicators

The project reports that, where monitoring and strategy development are not well connected, it appears that the performance management system will perform less well. If the indicators do not match well with the overall objectives, then management action in pursuit of the indicators is likely to lead to distorted outcomes.

A core set of outcome indicators (key and intermediate) is proposed, for use across the strategic decision-making process. The suite of indicators is drawn from those indicators already in use in the UK, but it provides a fuller coverage of sustainability issues than could be achieved by using just those mandatory indicators set out in Local Transport Plan 2 guidance. A method for prioritising the selection of these indicators is also proposed.

One conclusion of the project is that, because different areas have different priorities, not everyone will agree on one common set of indicators, and that recommending the adoption of indicators that are not relevant to local circumstances would be a futile exercise. Some indicators can be identified through top-down processes relating to key national sustainability policy principles. Others are better developed through stakeholder participation, as this can ensure that the most important aspects are measured and that the end users are clear about the outputs. The project describes a methodology for selecting sustainable transport indicators from a top-down perspective. It then applies this methodology using the definition of sustainable transport provided by the European Council of Ministers on Transport, and the currently available indicator sets used by local authorities in the UK. 10

This ECMT definition is divided into three parts, relating to society, the economy and the environment, and is as follows: -

“Sustainable transport,

For Society:Allows the basic access and development needs of individuals, companies and societies to be met safely and in a manner consistent with human and ecosystem health, and promises equity within and between successive generations.

For The Economy: Is affordable, operates fairly and efficiently, offers choice of transport mode, and supports a competitive economy, as well as balanced regional development.

For The Environment: Limits emissions and waste within the planet‟s ability to absorb them, uses renewable resources at or below their rates of generation and uses non-renewable resources at or below the rates of development of renewable substitutes while minimising the impact on land and the generation of noise.“11

The motivation for Project G of the DISTILLATE Programme, “Enhanced Appraisal Tools“, came from local authority concerns about appraisal, and their perception of appraisal as a barrier to the development of sustainable transport and land use policies. The outputs of interest, to QUEST, from this DISTILLATE project are the tools used to assist in decision making. The final product took the form of an assessment matrix which allows users to do the following: -

select a set of indicators appropriate to the context of a study

10 Marsden G, Kelly C, Snell C and Forrester J, DISTILLATE Deliverable C1: Sustainable Transport Indicators – Selection and Use (December 2005).

11 Council of the European Union, 2001.


03/01/2012 44 TTR

weight the indicators in terms of importance

assess the impact of proposed projects against the indicators

combine the weighting and assessment to derive a final score that will give an indication of whether a project is worth pursuing, OR, where more than one project is being considered, produce a priority list.

The project also provided a pick list of potential indicators derived from NATA (New Approach To Appraisal), Local Transport Plan guidance, Best Value indicators and other appropriate sources. Some flexibility was allowed, to enable users to add their own local indicators, to reflect specific local issues in a user„s LTP, or regional strategies. Indicators are likely to be weighted in line with the LTP, but different priorities can be set, if appropriate. Guidance on selecting indicators, weighting and assessing the impact of projects was provided to accompany the matrix. Likely users of this matrix include local authority decision makers, although it could also be used as part of the consultation processes with stakeholders. In the first instance, the matrix is designed to support the decision making process for small schemes, but it is also intended that it can be used with larger schemes to provide an initial indication of whether they are worth pursuing to the full appraisal stage.

4.3.8 OPTIMUM II (Optimal Planning Through Implementation of Mobility Management)

OPTIMUM II was an INTERREG III (North West Europe) project which ran from 2008 to 2010. It involved Local Authorities from the UK and The Netherlands, and aimed to substantially increase the effects of Mobility Management by focusing upon five key factors which the project labeled “The OPTIMUM II pillars“; these are: -

adopting a user-oriented approach

incorporating Mobility Management from the start

communication and information

marketing and promotion

enforcement of Mobility Management measures.

The project‟s main contribution is a set of guidelines for improved implementation of Mobility Management within the planning process of urban areas, especially in relation to land-use planning. The project appears not to have produced a specific approach for assessing and evaluating the results of this planning process.

4.3.9 MOBILIS (Mobility Initiatives for Local Sustainability)

MOBILIS was a CIVITAS II Project, which took place from 2005 to 2009. It involved CIVITAS MOBILIS partner cities – namely, Toulouse (France), Debrecen (Hungary), Ljubljana (Slovenia), Venice (Italy), and Odense (Denmark) - and their local mobility stakeholders. The project‟s aim was to strive to create a culture for clean urban mobility within the framework of sustainable development, ensuring involvement of all relevant stakeholders and participation of citizens. MOBILIS implemented radical strategies for clean urban transport in all five cities, building on a broad range of policies. The project used a total of 46 measures, which addressed all eight CIVITAS policy fields: -

alternative fuels and clean vehicles

collective transport and intermodal integration

demand management

influencing travel behaviour

safety, security and health

innovative mobility services


03/01/2012 45 TTR

energy efficient freight logistics

transport telematics.

These measures were integrated into a detailed work programme over the project duration of four years. MOBILIS concluded that a real change towards true co-modality could only take place through a convincing set of integrated policies. This included land use, mobility planning, social policies, health policies, and alternative mobility favouring actions, each adapted to the “unique” local context.

Evaluation concepts in MOBILIS were consistent with the GUARD approach, (see sub-section 4.3.3), as described in document D.2.1 “Framework for Evaluation” (2006). The MOBILIS Evaluation Plan consisted, for each measure, of the following: -

evaluation of functionalities

evaluation of the technical impacts (also referred to as Impact Evaluation)

evaluation of acceptability

collection of cost-benefit data

process evaluation

consideration of gender issues.

Monitoring and management functions were carried out by mobility work groups, during the implementation of SUMPs.

Based on the MOBILIS experience, a focused set of policy recommendations was developed for local and European politicians and policy makers, to help each European city to realise a sustainable urban mobility culture. Recommendations for local policy makers included the following: -

set realistic but ambitious objectives

engage citizens in mobility planning and implementation

learn by doing, share experience and contribute to inclusive governance in sustainable mobility planning and implementation

creation of a local mobility work group, as this facilitates the realisation of a SUMP e.g. The MOBILIS experiment in Debrecen proved that the setting up of a mobility work group, bringing together all authorities with mobility competences, helps to develop a strategy, and implement the action plan. Annual monitoring of progress will also help to manage its implementation

create the financial investment in a SUMP

encourage the integration of mobility planning and management.

Recommendations for EU policy makers included: -

introduce a formal CIVITAS quality label, so that CIVITAS should become such a quality label for cities that adopt an integrated approach towards a more sustainable urban transport system. A key attribute for this quality label could be the presence of an integrated approach and the implementation of an on-going sustainable mobility improvement strategy. Periodic assessments should lead to a reconfirmation of the label over time. This will enable local politicians to promote its adoption, and communicate the need for a new urban mobility culture. It could be a type of “Agenda 21” for urban mobility.

4.3.10 MOVE project

The purpose of MOVE, an IEE project which ran from 2006 to 2008, was to select tools, and establish methods and standards, that could be used to achieve changes in attitudes and behaviour, which could in turn lead to a change to less energy intensive transport modes.


03/01/2012 46 TTR

The project remit was to make such a selection based on existing European initiatives. Another important objective was to encourage more European communities to work on mobility issues and to offer alternatives to the use of fossil fuels, so leading to a reduction in greenhouse gas emissions. MOVE featured the participation of seven European energy agencies: the Local Agency for Energy of the Province of Chieti (Italy), the Energy Agency of Plovdiv (Bulgaria), the Energy Centre Bratislava (Slovakia), The Energy Agency for Southeast Sweden, The Basque Energy Board (Spain), the Tipperary Energy Agency (Ireland) and Klimaschutzagentur Region Hannover (Germany).

The project‟s objectives were achieved by: -

providing methods that would help communities in their work towards less energy intensive transport modes

providing good examples and success stories that would encourage more actors to work with mobility issues

showing reductions in greenhouse gas emissions from individual projects in different countries, using the tools developed by MOVE

illustrating targets that were able to be reached

showing the costs and health benefits that would follow as a result.

One of the first steps of the project was the creation of an Energy Balance. This provided facts and figures, which included energy consumption and emissions data related to transport in the area. Secondly, the Energy Agencies were also required to set up a Virtual Mobility Agency within their own agency, and set up a steering group that would help to drive forward local projects.

Finally, after implementation of local projects, the SUMO methodology (System for EvalUation of Mobility PrOjects) was used for evaluation; (see sub-section 4.3.5). Using this method, an evaluation was carried out of the following: -

the Approach to the Energy Balance, (i.e. the use of standard data, as opposed to a software package)

the Impact for each Virtual Mobility Agency.

SUMO was used for evaluation at every stage of the project. At the time, this was a new methodology, and so there was much learning and re- learning involved, as each of the local projects progressed through the implementation stages. Although this process was described as being time consuming and difficult to comprehend - one of the unexpected outcomes of following the SUMO methodology was the large time resource required in carrying out surveys - the majority of the MOVE partners did find the process useful. They found that it gave clarity and direction to the project, and that it was a logical process. Because SUMO forces the user to evaluate every step of the process, from start to finish, it was, reportedly, very easy to detect whether the local project were on-track or off-track. It was considered to be a useful methodology to use in a group project because it allowed partners to share experiences, to benchmark experiences at various levels and to know whether they were on or off course in achieving the goals set. The methodology, and its corresponding target flow charts, were regarded as being a particularly useful tool for systematic evaluation and monitoring throughout the lifetime of the evaluation process.

4.3.11 CATCH (Clean Accessible Transport for Community Health)

CATCH was a Demonstration Project funded under the LIFE-Environment Programme. It concluded in 2005, with the publication of a Best Practice Guide for the Clean Operation of Buses. The results from CATCH have also been reported in three separate documents: a non technical summary of key outcomes, a best practice guide and a technical evaluation of the whole project. The project supported the EC's Sixth Environmental Action Programme by


03/01/2012 47 TTR

promoting sustainable mobility in order to improve air quality. CATCH was implemented in Liverpool (UK), Suceava (Romania) and Potenza (Italy).

The project adopted an innovative and multi-disciplinary approach to reducing air pollution in Liverpool (UK), driving forward and demonstrating both hard measures, such as the provision of clean buses and new bus services, and soft measures, such as personal travel plans, calorie mapping and the introduction of new transport policies. The CATCH project contributed specifically toward the design phase of some of the walking and cycling initiatives contained within Liverpool‟s City Centre Movement Strategy. Specific initiatives to which CATCH contributed were wider street crossings, on-road cycle routes and phased pedestrian priority signals. Better signage and enhanced pedestrian areas were also part of the package of works that were implemented to encourage active travel, and thereby reduce air pollution. There was also the introduction of an Air Quality Action Plan; (at the time, the City of Liverpool was being designed to accommodate freer-flowing public transport, less vehicular traffic on certain routes and the promotion of healthier forms of transport). Related schemes were also implemented in Romania (Suceava) and Italy (Potenza), and the results were used for best practice guidance throughout Europe.

CATCH was evaluated through advanced pollution monitoring techniques, and the impact on community health was evaluated using a Health Impact Assessment (HIA) methodology. This is relevant to SUMPs in terms of the use of HIA to evaluate the effectiveness of each demonstration. Feedback from cities involved in CATCH has influenced at least one UK Local Authority, working with a private sector bus operator, to install particulate traps on its buses.

Recommendations of the project included the following:

clarify the policy guidance linking transport and land use planning

adopt a partnership-based approach to transport-related environmental policy

encourage politicians to lead the process, by providing support for a vision to which the project can work

produce plans for long term funding for transport

formalise the link between Air Quality Action Plans and Local Transport Plans (i.e. SUMPs)

develop a formal industry standard for the presentation of comparative emissions.

4.3.12 PROSPECTS (Procedures for Recommending Optimal Sustainable Planning of European City Transport Systems)

This project was undertaken as early as 2000 to 2002, and was funded by the DG for Energy, Environment and Sustainable Development; PROSPECTS was part of the Land Use and Transportation Research Cluster. Its remit was to provide cities with advice and guidance on optimal land use and transport strategies for achieving sustainability goals, and, whilst there is no specific mention of SUMPs, the project is an important source in terms of providing an over-arching background reference on the evaluation of sustainability policies and measures. It is also a source of reference for informing the selection of indicators for such an evaluation.

The objectives, indicators and decision-making requirements of six core European cities – Edinburgh, Vienna, Oslo, Stockholm, Madrid and Helsinki - were reviewed in depth, and this review was supported with a subsequent survey of 100 additional cities. The project also assessed and enhanced evaluation procedures and optimisation methods.

Outputs from the project include a Decision-Maker‟s Guidebook and supporting Methodological and Policy Guides. One specific deliverable of PROSPECTS is a 150-page report on evaluation methods in relation to sustainability. This document includes lists of indicators which may be used for assessing the sustainability of urban land use and transport strategies. These indicators are listed under various categories, including Economic


03/01/2012 48 TTR

Efficiency, Increased Freedom of Movement for Vulnerable Road Users, Energy Use & Climate Change, Protection of Heritage Sites, Urban Sprawl, Noise, Equity & Social Inclusion, Traffic Accident Reduction, etc. As well as existing as a digest of indicators, the document includes a discussion of the indicators that are most likely to be used in planning exercises. There are five such indicators in the category of Environmental Indicators.

4.3.13 CO2NeuTrAlp (CO2-Neutral Transport for the Alpine space)

CO2NeuTrAlp is a project funded from the European Territorial Cooperation / European Regional Development Fund. It explores the feasibility of locally-sourced renewable energy being used for both private and public transport, as an alternative to fossil fuels. The project has a particular geographical focus on the Alpine Region, which comprises five countries. The work is based on the demonstration and testing of 13 pilot projects. These pilot projects encompass sustainable transport in both rural and urban contexts, and also include both passenger and freight transport.

The key guidelines from the project are contained in “Renewable Energies in Transport: Local Action to Promote the Shift towards the Age of Solar Mobility”, which is available as two separate documents – “Guidelines for Decision Makers”, which is aimed at decision makers in local and regional authorities, and “Guidelines for Transport Professionals”. These documents provide practical advice on how to plan and implement sustainable transport and mobility services.

Whilst there is no specific reference to SUMPs in either document, there is practical advice on how to involve stakeholders in the planning process. A “participative planning tool” is provided, with precise guidance on how to involve important stakeholders in the implementation of new, sustainable services. This includes a description of an eight-step procedure for establishing a co-operative Local Implementation Network. Evaluation of the pilot projects featured is undertaken using the MaxEva Evaluation Tool, (see sub-section 4.3.5).

There is a detailed discussion of criteria that should be used to inform decision making with regard to the use of renewable energy sources, and this is effectively a discussion of indicators that should be used for monitoring and evaluation. Among the criteria recommended are the reduction of greenhouse gas emissions, the net reduction of toxic emissions and the elimination of unnecessary trips. Criteria recommended are grouped under the headings: Environmental, Technical, Economic, Social and Spatial Development.

4.4 National reports on the evaluation of SUMP and similar planning documents

4.4.1 Mobility Handbook provided by the Flemish Government

This handbook was published by the Flemish Government, with the involvement of the Department of Roads and Traffic, the public transport operator for the region (De Lijn) and the Provincial Government, as stakeholders. The handbook contains guidelines on the process and content of a SUMP, and issues concerning funding, and also provides advice on the implementation of measures, and consultation with stakeholders.

There is an important distinction made here between process evaluation, which is concerned with the quality of the process of creating the SUMP, and impact analysis. For the process evaluation, documents are presented for review to the Provincial Audit Committee (PAC). This control organisation consists of an external independent auditor and the group of stakeholders. At the first stage, the PAC checks whether the appropriate actions for analysing mobility management issues have been identified, and whether sustainable scenarios have been planned. At the final stage, the PAC verifies the “Policy Plan” according to whether the plan: -


03/01/2012 49 TTR

is structured in the required way

contains all necessary elements, such as clearly defined objectives, an Action Plan, measures relating to land-use development, measures for all relevant modes of transport, an evaluation and monitoring plan, etc.

has been the result of a participatory process involving stakeholders.

The review of the impact analysis involves an overview of monitoring actions and evaluation analyses; it also checks that there are meetings scheduled with stakeholders, and that there is a timetable in place for updating the Action Plan.

What does not appear to be adequately covered in this review process, however, is how the quality of the plan‟s measures is evaluated, in terms of, for example, whether they address the appropriate issues and themes. Further work is also required on the impact analysis, in setting clear objectives for planned actions (i.e. quantifiable targets) and developing indicators for evaluating progress.

4.4.2 Guidelines for consultation on Urban Mobility Plans (CERTU)

The French research organisation CERTU, (the Département Mobilité et Déplacements), published a document, in 2003, entitled “La concertation dans les PDU: Pourquoi ? Avec qui? Comment?”, which means “Consultation in SUMPs: Why ? With whom ? How ?”. This publication is a set of guidelines for setting up public consultation procedures for PDUs (Plans de Déplacements Urbains), the French equivalent to SUMPs, adapted to the local context. The document contains information sheets on the different tools that can be used for providing information, building a dialogue and monitoring public opinion. This document sums up the most important aspects of a public consultation, identifying the relevant elements for choosing the most appropriate form of public consultation, (i.e. which stakeholders, representativeness, whether to consult the public or organisations, which levels of a public authority to consult, etc.).

These guidelines also focus on evaluation of the process, specifically on the requirements for public consultation. The French law LOTI (Loi d‟Orientation des Transports Intérieurs) specifies a number of requirements for the creation of a SUMP, including: -

the number of actors to be involved

consultation with public bodies

consultation with the affected population.

To evaluate the impact on the environment, an environmental report must be provided, and this should also be subject to the consultation procedures described in the document. However, the advice of public bodies consulted is not binding, nor is it obligatory for adjustments to be made to the SUMP in response to advice on the environmental impact of the plan.

Whilst the document provides clear guidelines on the process of consulting with both organisations and members of the public, procedures for the evaluation of the impact of a SUMP are not yet clearly defined. The guidelines do, however, state that there is need for clear objectives for the planned actions, (i.e. quantifiable targets), and that clear indicators should be identified for evaluating progress.

4.4.3 Guidance on Local Transport Plans provided by the UK Department for Transport

In 2009, the UK Department for Transport published a guidance document outlining a policy framework for Local Transport Plans (LTPs), which are detailed plans that Local Authorities are legally obliged to draw up, and which stand as SUMPs in the UK. The document describes key steps in developing and delivering quality LTPs. There is no clear process or method of evaluation of LTPs outlined, although reference is made to indicators identified in


03/01/2012 50 TTR

the DISTILLATE Project (see sub-section 4.3.8), and this project‟s guidance on the development of a monitoring strategy and the selection of indicators. The document states that Local Authorities should consider, as they develop their Plan, what performance indicators are most appropriate for monitoring it, and what targets might be set to incentivise and secure delivery; performance monitoring should be an integral part of managing the LTP programme. The document also suggests that a strong LTP will include ambitious target setting, clear trajectories and close monitoring of delivery. Local Authorities are encouraged to discuss with other authorities, especially within their region, what standard indicator definitions may enable them, and the wider transport community, to benchmark their performance. Evaluation needs to be planned carefully before SUMPs are taken forward, so that processes can be put in place to collect the necessary data. Results should be shared across organisations responsible for SUMPs, to help build a knowledge base. There is some emphasis on the importance of using indicators, and on the exchange of results with peers.

4.4.4 The National situation in the Czech Republic

There are two national strategies in the Czech Republic which deal with sustainable transport in urban areas. These are “Transport Policy for the Czech Republic 2005-2013” and “General Plan for Transport Infrastructure Development (GEPARDI)”. It is the first of these documents that refers most to urban transport plans. This plan was drawn up by the Czech Ministry of Transport, in cooperation with other relevant ministries, and has been updated on two occasions, the last of these being in 2011. Each update has been preceded by a transport policy assessment study. The successor document, “Transport Policy for 2014-2020, with outlook to 2050”, will be developed from 2012. The policy was drawn up with the involvement of experts from universities, NGOs and representatives of the private sector, through a series of stakeholder workshops.

Formal evaluations of the policy have taken place using the Systematic Environmental Assessment (SEA) methodology, focusing particularly on environmental and public health issues. Environmental evaluation has included an assessment of impacts in Natura areas, (parts of the Czech Republic that belong to the EU‟s network of nature protection areas), whilst public health impacts of different policy interventions have been categorised using a scale from +2 to -2. Further analysis has been carried out using the Life Cycle Assessment (LCA) evaluation method, considering the impact on the environment and public health of, for example, 1 km of new road, or additional rail or water infrastructure.

Use of the SEA method of evaluation is considered to have worked very well, with clear and transparent evaluations being provided. The stakeholder workshops have also been deemed to have been successful, in the way that they have stimulated discussion between the creators of policy, experts and the general public.

4.5 Evidence from interviews with urban mobility practitioners

4.5.1 Summary of findings from interviews with urban mobility practitioners

A small sample of urban mobility practitioners was interviewed, in order to gain feedback as to their personal experiences with SUMPs. This sample represented cities with different levels of familiarity with the SUMP process. The cities also displayed different approaches to evaluation, from precise annual evaluation regimes involving a number of set goals, to no evaluation at all, due to being at the beginning of the SUMP process. Typically, cities already conducting evaluation used a combination of annual reviews, allied to more periodic in-depth surveys (every four to five years, for instance).

Generally, respondents stated that there was some involvement with local stakeholders, sometimes with stakeholder workshops being held, and cities generally conducted some form of sustainable urban mobility planning, even if this was not in the form of a separate and formal SUMP, but part of a wider planning process.


03/01/2012 51 TTR

Reflections on experiences with the SUMP process, and recommendations for others wishing to plan and implement a SUMP were varied. Whilst some of the respondents appreciated the role and value of monitoring and evaluation, others pointed out that it is a costly and time-consuming process. There was general agreement that it is important to define clear goals and objectives, but that there should be an appropriate number of these, so that the task of data collection and analysis does not become unmanageable.

4.5.2 Characteristics of the sample

Seven urban mobility practitioners were interviewed, and these represented a variety of roles; they were a Head of Traffic Planning, a municipality‟s Strategic Planning specialist, a Climate Strategist with responsibility for traffic issues, a representative of a municipal Mobility Planning Department, a Mayor with particular responsibility for urban development and transport, a Transport Policy Adviser from a Think Tank / Research Institute who had experience of providing advice on sustainable transport issues for a number of cities, and a researcher specialising in the environmental impacts of transport. The latter respondent provided information on guidelines and legislation at a national level in the Czech Republic, and this information has been presented in sub-section 4.4.4. These six interviews which focused on specific cities referred to cities in four countries: Sweden (the cities of Lund and Örebro), Belgium (Gent, which is in Flanders), the Czech Republic (Hradec Králové) and Germany (Freiburg and Tϋbingen12).

Whilst this sample has a strong North European bias, it nevertheless represented a range of levels of experience with the creation and evaluation of SUMPs. For instance, formal Sustainable Urban Mobility Plans are very new in Germany – in contrast to France, Italy and the UK, where there is a statutory requirement for such plans. The respondent from Freiburg did, however, point out that the city had started to think in terms of “alternative mobility” some 40 years ago, with planning for a number of alternatives to the car being considered. At the time, the policy was not labeled as “sustainability”, but was referred to as “stadtverträglicher verkehr”, which translates approximately as “city compatible transport”. The city now has a Mobility Development Plan, but this was not created until 2008. Plans are already in place for the next steps in the process, which will include further developments for cycling in the city, traffic safety improvements and moves to encourage modal shift to more sustainable means of transport. There are no official national guidelines available in Germany on SUMPs, although there are technical guidelines available on transport and mobility planning in general, at an urban area level, from the “Forschungsgesellschaft Stadtverkehr”, which is the urban transport research organisation in Germany.

Sustainable transport planning appears to be a little more advanced in Tϋbingen, however – but it is suggested that this has been driven by the current Green Mayor. The city has a Strategic Plan in place for the period 2010 – 2030, and this plan includes mobility management for businesses, improvements to both public transport and public spaces in order to reduce the use and impact of private cars, and the development of “public bike” schemes and the use of electric cars.

In contrast, the City of Gent has had a Mobility Plan in place for the city centre since 1997, and for the whole city since 2003. There has been an on-going process of updating the current plan since 2008, and this is expected to be completed during 2011.

In the Czech Republic, as in countries such as Poland, Slovakia and Hungary, there is no preparation of a separate SUMP, but sustainable mobility planning is one element of a detailed planning process. The city of Hradec Králové created a Strategic City Development

12 Whilst the respondent had experience of working with several cities, responses were provided on the basis of experiences with Tϋbingen.


03/01/2012 52 TTR

Plan in 2004, which was in accordance with the principles of the Republic‟s Local Agenda 21. Transportation is one of eight key areas featured in this plan, and consists of five themes: 1. The city‟s public transport network; 2. Connections with national public transportation networks; 3. The function of Hradec Králové Airport; 4. Non-motorised public transport (i.e. walking and cycling); 5. Minimising the impact of transport (e.g. parking measures). Specific goals and indicators will be associated with each of these themes, and these are bound together in an Action Plan which both supports the overall objectives of the Strategic City Development Plan, and includes details of specific activities and projects.

The situation reported by the respondents from the two Swedish cities underlines the fact that levels of awareness of, and experience with, sustainable urban mobility planning can vary considerably within a country. The first SUMP for Lund was established in 1998, after having been discussed in the city since the mid-90s. (This was actually referred to as a “plan for an environmentally adapted transport system”). The focus of this plan was environmental sustainability, but efforts have been made in subsequent plans for there to be a greater emphasis on social and economic sustainability. This was certainly the case with the second SUMP, which was produced in 2006. The next plan for Lund is currently under construction, and is due to be delivered in 2013; the focus of the new plan is on planning process, rather than on projects. Generally, it is Lund‟s policy to create a mind-set that considers planning for sustainability to be an every-day basis for traffic planning.

Örebro, on the other hand, did not have a Transport Plan recognisable as a SUMP until 2008. Prior to this, consideration of sustainability issues in transport had been very ad hoc. Some projects to encourage cycling had been undertaken, and the city had taken part in the BUSTRIP project, but this fragmented approach suggests a low level of policy development in this area.

4.5.3 Approaches to evaluation

Just as there was evidence of considerable differences in the level of experience with SUMPs within the sample, there were also marked differences in the approach to evaluation between the cities represented.

Lund has a fairly precise evaluation procedure, which is conducted internally, on an annual basis, with a report published each year. This is founded upon 18 set goals. In addition, an extensive public attitude survey is carried out every four years. This survey monitors public awareness of the sustainability agenda and the city‟s SUMP, and people‟s perceptions of changes that might have taken place in the local environment. It is estimated that from half a person-month to one person-month of resource is consumed by the annual evaluation of the SUMP. There is no external input to this evaluation. Örebro also carries out an internal annual sustainability evaluation, using a number of traffic indicators, as well as having additional periodic travel surveys (the last one having been in 2004), on-going monitoring of car and cycle usage (using automatic data capture equipment on set routes) and public transport ridership data. It is estimated that data collection and analysis in relation to the SUMP evaluation process requires about one person-month of resource. Again, no external evaluation of the transport plan has so far been carried out.

Hradec Králové conducts an evaluation annually. Indicators, which are a mixture of European Common Indicators and those which the city has developed, are updated. The entire Strategic City Development Plan will be updated in the Autumn of 2011. The city follows a plan for consulting stakeholders that was established during the creation of the Plan. This stakeholder group includes businesses, NGOs, experts and students. It is estimated that the city spends €12,000 per year on research and analysis, and in obtaining data on indicators, in connection with the Strategic Plan.

The initiative for creating a SUMP for Gent came from the City authority, although the process is now based on a partnership that includes the Flanders Regional Government, the


03/01/2012 53 TTR

Provincial Government and the public transport operator for the Flanders Region (De Lijn). The City of Gent follows an evaluation methodology that has been determined by the Flemish Government. This consists of the following steps: -

1. A review of the current mobility plan – for Gent, the finding of this review was that updates were required for some themes

2. Updates made, where necessary

3. Completed mobility plan, which is valid for five years

A total of €200,000 of funding was set aside, some two years ago, for the SUMP process in Gent, which is expected to be concluded by the end of 2011.

Because the formal process of SUMP creation is less well-developed in Germany, SUMP evaluation is also less well-defined. However, in Freiburg, monitoring of the city‟s transport plans is carried out internally, but this is focused on public transport passenger numbers and accident statistics, etc. in relation to set targets. There is no budget set aside specifically for this monitoring. Stakeholders are involved in the planning and development phases, but not in evaluation and monitoring activities. Similarly, a review is carried out of Tϋbingen‟s Strategic Plan every five years – but there is no SUMP evaluation, as such. Again, the involvement of stakeholders is confined to the planning and development phases. This was achieved by means of three stakeholder workshops, each of which was attended by 30 to 50 people. Invitations to the workshops were issued to major employers, business representatives, political parties and local user groups.

4.5.4 Pros and cons of different approaches to Sustainable Urban Mobility Planning, and lessons learnt

Respondents were asked to reflect upon the aspects of their city‟s sustainable urban planning procedures that they think have worked well, and on those that they feel have worked less well. They were also asked about lessons learnt from the process, and also for recommendations they would make to other cities embarking on creating a SUMP.

Specifically in relation to Gent, it was felt that the procedure provided for the review of the mobility plan might have been inappropriate for a city the size of Gent; (population of approx. 243,000). It was also felt that the five-year time horizon for the plan allows just enough time for actions to be started. The main positive aspect of experiences in Gent was the opportunity for all employees within the City‟s Mobility Department to be involved in the planning process, and the flexibility that the Department had to determine the detail of the process, within the methodological guidelines set by the Regional Government.

General advice and recommendations offered by our six respondents are listed below, in no particular order; (where recommendations appear to be contradictory, this merely reflects differing views provided by our respondents): -

an annual review and evaluation of goals and indicators is useful, as this will highlight the areas that need attention

preferably, goals should be selected that are shared by other cities, as this will enable comparisons to be made.

whilst monitoring and evaluation of a plan are important, it should be borne in mind that this is a costly and time-consuming process.

cities should plan and budget for carrying out potentially costly data collection, in order to be able to justify expenditure on the sustainable mobility planning process.

obtaining data for the required indicators can be both costly and time-consuming, and then the results of the evaluation process might not have much influence on decision makers.

co-operation with the private sector is important in the process of data collection.


03/01/2012 54 TTR

it is a basic prerequisite for a SUMP that politicians should be willing to respect its objectives and priorities.

the importance of evaluation should not be over-estimated, and not too much resource should be put into this activity. Data that are reliable and objective are difficult to collect.

evaluation is important, as it provides feedback, and so ensures that a plan is a dynamic document that reflects current trends in a city.

the most important thing is to define clear goals, which are readily measurable, and which can be monitored using existing statistics. This is to minimise the cost of the monitoring and evaluation process. However, the priority is that the indicators selected should support wider planning goals and objectives, and not be chosen merely because they are easy to collect. Experiences to date have shown that some indicators have been both appropriate to wider goals, and associated with readily available data – but others have largely failed on both counts.

it is important to define an appropriate number of goals, since too many will impose a resource burden on the authority in the future, and potentially make it difficult for the appropriate data to be collected. It is recommended that the ideal number of goals to set is between 10 and 20.

it is important for there to be coordination between transport policy and land-use planning policy, since the SUMP needs to consider issues relating to local housing plans, neighbourhood development, etc.

too much emphasis should not be placed on climate change issues, since it is better to emphasise the the advantages of improved access, (as a result of reduced congestion, for example), for people, and benefits for the local economy. This is because these impacts are easier to “sell” to both politicians and the public at large. Successes in relation to CO2 mitigation, for example, should be regarded as an “extra”.

CO2 reduction targets can be very challenging in cities where environmental standards are already fairly high. In such circumstances, it can be difficult to demonstrate positive environmental impacts, once “the low-hanging fruit have been picked”. It might be advisable, therefore, to place an emphasis on seeking social and economic benefits.

the maximum life-span of a SUMP is about ten years, after which there is a requirement for re-prioritisation, and for the acquisition of new knowledge.

whilst it is good to learn from other cities which have more experience of the sustainable mobility planning process, it is important for the creation and evaluation of the SUMP to reflect the local context.

cities undertaking a SUMP for the first time should be prepared for there to be a considerable resource requirement initially – but this requirement will reduce, over time.


03/01/2012 55 TTR

5 Comparison of the approach of QUEST, with that of other projects using TQM to develop audit tools

The objective of task 3.3 was to provide an overview of previous projects which have used the concept of TQM in order to develop an audit tool. This review has featured projects whose methodology and approach have a great deal in common with the approach of the QUEST Project, and this chapter describes the synthesis of evidence compiled in order to make recommendations for the development of the QUEST methodology and tool. It is not an aim of this report to provide a complete overview of all quality management and audit schemes, but the results of the state of the art analysis of two STEER projects - ECOMOBILITY SHIFT and ISEMOA - have been explored. Both projects have made an exhaustive inventory of exiting quality management schemes.

In order to systematically collect information about quality management and auditing schemes, an inventory has been prepared (see Appendix 5). In the project preparation phase, the projects BYPAD, MaxQ and Mediate, which have used the concept of Total Quality Management, were identified as a major inspiration for the development of QUEST. Therefore, these audit methods have been explored thoroughly. In addition, there was consideration of the Flemish PAC project and the Civitas Pointer project. The latter has been inspiring on process and impact evaluation. Together with the recommendations of SHIFT and ISEMOA, it has been possible to distil strong guidance for the development of the QUEST tool from these sources.

The goal of the QUEST project is to enhance the level of quality of the transport system in small and mid-sized cities, and of the processes that precede achievements in practice. Therefore, the main questionnaire: What is necessary for enhancing sustainable mobility ? How can this be done ?

5.1 Quality management schemes

SHIFT explored 25 national and international tools and methods in order to improve quality in organisations.13 The ISEMOA project covered 56 projects, so a substantial range of audit schemes has been thoroughly explored.14 This review focuses on relevant information for the development of a TQM scheme in public administrations with a focus on the transport sector.

5.1.1 Total Quality Management in public administrations

Quality management has evolved throughout the last three decades, from simple inspections, quality control and quality assurance, to modern, comprehensive philosophies and approaches such as TQM, ISO 9000 and EFQM/CAF. Today, quality is commonly seen from an integral, organisational and consumer-oriented perspective: quality is a dynamic state associated with products, services, people, processes and environments that meets or exceeds expectations. The critical, underlying meaning of the above statement is that quality is not just about results (vehicles, level access), but also addresses the people (authorities,

13 Quental N, Kirkels M, ECOMOBILITY SHIFT Deliverable: An EcoMobility certification system for cities. Technical brochure (May 2011)

14 Bösch S, Hyllenius Mattisson P, Fredriksson L, ISEMOA Deliverable D2.1: Compilation of QM-schemes and audits (March 2011)


03/01/2012 56 TTR

operators) and processes (information, training) involved in producing these results. Quality is achieved through quality management, a systematic set of activities followed by the organisation in all its facets, aiming at maximising customer satisfaction (which might be manifested in the form of the opportunity to choose sustainable modes of transport). A TQM-oriented organisation is characterised by a strong consumer-oriented approach, a commitment to the elimination of errors and steps that do not add value to products and services, a firm focus on prevention, long-term planning, teamwork, fact-based decision making, a continuous pursuit of improvement, horizontal and decentralised organisational structures and external partnering arrangements.

Originally developed to improve process, various approaches have been developed in order to support the public sector. Public managers became aware of the initiatives that had been proven to be sound in the private sector, and many aspects, concepts, techniques and methodologies originating from the private sector were adopted. This tendency is generally referred to as the New Public Management movement (NPM). As a consequence of the growing attention for quality management in public administrations, the need for a supporting quality framework emerged. Most of the tools used in the public sector today stem from instruments used in private organisations that have been adjusted for the specific characteristics of the public context.15

5.1.2 TQM in the transport sector

Quality management schemes in the transport sector which have been adjusted according to the needs of policy process are BYPAD, MaxQ and Mediate. These schemes are similar to one other, as they are all inspired by the general TQM tools ISO, EMAS and EFQM. These approaches describe actual situations, and audit the working process in the relevant entity. All these tools aim to support progress by giving organisations tools for continuous improvement, based on the „Plan-Do-Check-Act‟ principle.

BYPAD and Mediate are sectoral approaches in the specialised fields of cycling policy and the accessibility of public transport, respectively. MaxQ, as a quality management system for Mobility Management, has no special focus, but covers all elements of mobility. BYPAD has been developed and refined in the past decade, and has been implemented in more than 100 cities in 21 countries. Mediate has recently been developed, in 2010, and tested in two regions (Lisbon and Flanders (De Lijn)). The MaxQ approach has been successfully used in Sweden (ISEMOA 2011).

The structure of the approaches is similar. Policy is seen as a dynamic process where different components must fit together in order to achieve a well-balanced policy cycle. The components of the dynamic process are divided into different elements, or modules. The number of components and elements varies in the three approaches (Table 1).

Table 1 Components and elements of QM schemes in the transport sector

BYPAD MaxQ Mediate

Components Elements Components Elements Components Elements

Planning User Needs Policy User Needs Planning User needs

Leadership & Coordination

Policy on paper Leadership

Policy on paper Leadership Guidelines &

15 De Jong M, Sweers W, Mediate Deliverable D4.1: Mediate – Methodology for Describing the Accessibility of Transport in Europe (2009)


03/01/2012 57 TTR

Policy on paper

Means & Personnel

Strategy MM Programme Resources & Personnel

Action Infrastructure & Safety

HR management

Implementation Vehicles & Built Environment

Information & Education

Partnerships Information & Ticketing

Promotion & Partnership

Budget Training & Education

Complementary actions

Implementations MM Measures Seamless Travel

Monitoring Evaluation & Effects

Supportive measures

Evaluation Results

Monitoring & Evaluation

User & Society results

Evaluation & Effects

Stakeholder feedback

Management review

As shown in the table, the components are quite comparable: Policy/Planning/Strategy (A), Action/Implementation (B) and Monitoring/Evaluation (C).

The elements are also comparable. All three approaches use the elements in field (A) Policy/Planning/Strategy: User needs (1), Leadership (2), Policy on paper (3), Personnel/HR management (4), Means/Budget (5).

In field (B) Action/Implementation Mediate and BYPAD use similar elements actions in field (6) and Information & Education (7). MaxQ describes element 6 as Mobility Management measures. Partnership is evaluated in BYPAD and Mediate (8). Otherwise, Complementary actions/supportive measures are evaluated in BYPAD and MaxQ.

In field (C) Monitoring & Evaluation, BYPAD defines just one element: Evaluation & Effects (9). In MaxQ, (C) is more divided in aspects User & Society results, stakeholder feedback and management review.

This policy cycle approach assesses vision, goals, programmes, and routines in the organisation for implementation and monitoring. The differences are based on two aspects: on one hand, the presentation of results, but mainly on the slight difference in approach of MaxQ compared to BYPAD and Mediate. BYPAD and Mediate work with the concept of self-assessment which is carried out by key stakeholders, or, in BYPAD, the evaluation group. The composition of the group is crucial to the success of the evaluation process. The stakeholder group consists of representatives from end user organisations (e.g. a cycling federation), and also of management and politicians. As SHIFT also concludes, it is a challenge for QUEST to make clear how all modes of transport (walking, cycling, public transport, car traffic, freight transport, etc) can be covered in field B Action/implementation.


03/01/2012 58 TTR

Table 2 Ladder of development in BYPAD, MaxQ and Mediate

BYPAD MaxQ Mediate

5 - Total quality MM

4 - Integrated approach

4 - Chain-oriented MM-approach

4 - Integrated approach

3 - System-oriented approach

3 - System-oriented MM-approach

3 - System-oriented approach

2 - Isolated approach

2 - Process-oriented MM-approach 2 - Isolated approach

1 - Ad hoc-oriented approach 1 - Activity-oriented MM-approach 1 - Ad hoc-oriented approach

0 - No evidence of MM-approach at

all

All elements are assessed separately using a ladder of development. This ladder defines the actual quality level of the different elements. Mediate and BYPAD use the same four-step ladder, while MaxQ uses 6 quality levels. Each level represents a further step on the ladder of development – thus representing further progress of the organisation‟s working process, and, to some extent, in outcomes. Quality improvement is achieved as the organisation climbs the ladder of development. The actual names of the different levels are less important, as long as the naming follows a logical order, and the levels are well defined (ISEMOA 2011).

The separate assessment of the different elements is recommended by SHIFT, where interviewees strongly recommended that a QM-model should show the results for the different elements separately, in order to increase transparency and understanding of the system (SHIFT 2011).


03/01/2012 59 TTR

Figure 1 Scoring different elements in the Mediate project

BYPAD also assesses the quality level of each element separately. The overall quality level of cycling policy is determined by the weighted average of the quality levels of the single elements. Mediate, which was inspired by BYPAD, did not use overall weighting, focusing on addressing improvement opportunities according to the different elements of the policy cycle, (as shown in Figure 1). The figure illustrates the assessed level of development for the 9 defined elements/modules of Mediate. For three of the elements, targets for improvement have been defined.

The Mediate approach makes benchmarking impossible, but for good reasons. Mediate wants to support organisations in making progress, with the assumption that each organisation can make progress by isolating crucial elements and developing feasible actions. Mediate thus focuses on strong choices for improvement. The scoring process is just an instrument for defining the status quo for each element.

BYPAD is designed to improve the cycling policy of a city/region, using an internal evaluation process, and learning from other experiences in European cities/regions. An overall score of the city‟s cycling policy is determined by the expert judgment of the local stakeholder group, which is in turn based on a combination of qualitative and some quantitative indicators. Of course, it is possible to compare the scores of the BYPAD cities in order to benchmark. The BYPAD evaluation16 makes clear that this tool is not designed as a “beauty contest“. Judgment scores are difficult to compare. It is informative to learn of how other cities are scoring, but it should always be taken into account that this comparison is not reliable.

16 BYPAD Deliverable D6.5: Cycling, the European approach. Total quality management in cycling policy. Results and lessons of the BYPAD-project (October 2008)


03/01/2012 60 TTR

The following issues should be taken into account when comparing different cities: -

Geography of a city

Different BYPAD auditors

The personal opinions of the people in the evaluation groups

Cycling culture

To enable a transparent benchmarking of cities, a clear set of quantitative indicators is required. In order to assess this information, a great deal of effort would be necessary. QUEST is not focusing on certification, but sets out to support cities in actually making progress. Therefore, QUEST will have a strong focus on the quality of the policy process (e.g. how can sustainable development strategies and improvement programmes be developed ?). Scoring should be seen as a tool for defining deficiencies in sustainable mobility policy, and for helping cities to climb up the ladder of improvement.

5.1.3 Assessment process and the role of the auditor

The way the assessment is organised and communicated to the stakeholders has an impact on the level of support provided. The external auditor‟s role in the process influences the willingness of stakeholders to participate in the process and support actions.

Crucial points are: -

involvement of local stakeholders in the audit process

self-assessment, as opposed to assessment by an external expert.

In all three methods, local stakeholders are involved. In BYPAD and Mediate, the external expert guides the city representative in setting up a group of appropriate stakeholders which should be in a position to define user needs, and to be capable of evaluating planning processes and outputs using expert judgment. These can be city officials from relevant departments such as planning, environment and health, representatives of transport authorities and local lobby groups for cycling or older people, etc.). ISEMOA also underlines the importance of new networks of relevant stakeholders which together safeguard the improvement process. It is unclear how the Mobility Management team (MM-team) is set up in MaxQ.

This involvement of stakeholders is a crucial aspect of TQM, in contrast to standard auditing. By involving local stakeholders in the assessment and improvement plan, it is more likely that improvements will be put on the agenda and really be implemented. Moreover, there is a higher probability that the proposed changes will be regarded as important, and be appreciated by the end users. Therefore, QUEST will focus strongly on the involvement of key stakeholders in making the action plan successful.

BYPAD and Mediate clearly focus on self-assessment. The tool is meant to help local stakeholders to evaluate the state of local policy themselves. The auditor guides stakeholders in evaluating their own policy, and supports them in developing an improvement programme. Therefore, Mediate does not refer to the person in question as an „auditor‟, but uses the term „moderator‟ for the external expert. In BYPAD and Mediate, the auditor/moderator develops the Action Plan based on the results of the self-assessment exercise and agreed actions.

Depending on the ambitions of the cities, current levels of quality, the level of political awareness and feasibility in financial terms, MaxQ developed 5 different assessment


03/01/2012 61 TTR

Track 1 „blue print for a kick

start‟

Track 2 „strengthening

urban mobility

Track 3 „inspiration for

excellence‟

Guide = External expert + Track 2 city

Guide = External expert + Champion city

Guide = External expert

Basic principles of urban mobility plan

Focus on 1 or 2 key themes Exchange of knowledge, hosting site visits

Initiating the development of an urban mobility plan

Getting inspiration from champion cities

Exchange of knowledge by staff exchange and visitation commissions

Getting inspiration from Track 2 and Track 3 cities

procedures: -

1. self assessment

2. small internal audit

3. internal audit

4. external audit

5. certification and benchmarking

The MaxQ self assessment is no more than a short, structured questionnaire for a quick scan of the quality status of the city, design to encourage stakeholders to take part in the deeper process. Therefore, the “self assessment” procedures used in the three approaches are not comparable. The internal audits are more exhaustive, but still entail no guidance or supervision. The external audit is comparable to BYPAD and Mediate, in as much as the external auditor or facilitator adds his or her own expertise to the overall validity of the process. Depending on the degree of assessment, MaxQ works with just internal auditors, with one external auditor, or even maybe with two external auditors, in the certification and benchmarking processes. There is no evidence that the “certification and benchmarking” level has yet been fully developed. According to ISEMOA, no organisation has asked for this so far (ISEMOA 2011).

QUEST intends to work with an external expert to guide the cities to improvement in sustainable mobility. The approach of MaxQ, with several audit possibilities starting with internal audits, is considered to be too time consuming, since cities with a poor level of quality management might take years before they can realise improvements. The internal audit does not give enough guidance for the cities which need the support the most. The main aim of QUEST is to support different types of city, with a strong focus on cities with currently poor performance on sustainable mobility (policy). Therefore, QUEST will develop a method where the external expert supports the self-assessment by the cities. It is still possible, however, to learn from MaxQ, since QUEST seeks to define three different tracks in the auditing process, depending on the level of development of the cities (Figure 2). It is feasible that the work required for carrying out a self-assessment as to the current level of development of policy for a Track 1 city is much shorter than that required for Track 2 and 3 cities. The focus here should be on how the development of urban mobility planning can be initiated, which measures are promising (based on best practice) and which stakeholders‟ participation is necessary for achieving improvements.

Figure 2: Three potential tracks of QUEST


03/01/2012 62 TTR

The tasks of the external experts can be summarised as follows: -

collecting information through bilateral contacts with the cities representative and analysing these documents

assists the city to set up a stakeholders group

using a questionnaire to assess stakeholders view on the elements of the policy cycle

several meetings with relevant stakeholders to assess the quality of mobility plans and their implementation

meetings with the stakeholders to agree on the assessment rates

meetings with the stakeholders to assess and agree on the input for the QUEST Action Plan.

These detail of these tasks needs to be defined in the context of the three defined tracks. The time taken for the external expert to complete tasks might also vary according to the level of development of the city in question. For all levels of development, however, the audit requirements from the external experts need to be determined beforehand, in order to safeguard the standard of the QUEST audit. Auditors will be trained in the QUEST ACADEMY . They will have to be able to guide the cities through the auditing process, and make decisions about the certificate that will be given to the participating cities. The body responsible for the designation of the QUEST auditors is, in the first instance, the QUEST consortium. The QUEST ACADEMY has to make clear how the certification of auditors will be organized, in the post project phase. SHIFT explored several guidelines on QM-systems which describe the knowledge and skills needed by certified auditors.

For QUEST, the external expert must be independent, in as much as he or she plays no role in the process that is assessed. The expert must have qualifications in quality management, and expertise in mobility planning, with a focus on sustainable mobility. On the one hand, the auditor must have skills and knowledge relating to the auditing process, procedures and techniques, and be able to stimulate participants into making a self assessment (SHIFT 2011). On the other hand, the auditor has to have significant knowledge about mobility planning in general, but also about decision making in public administrations and national legislations and guidelines (BYPAD 2006). The QUEST ACADEMY clearly needs to define the requirements of the certified auditors, and take these requirements into account during the training sessions.

5.1.4 Certification and benchmarking

The review found that the majority of schemes incorporate a form of certification, or grant permission for the use of a logo (e.g. ISO) to be used. In essence, a certificate is based on participation in the assessment process. A minority of schemes have a label based on a scoring system. Labels have been introduced in order to provide motivation to their users to strive for further improvement. CAF, for example, bases its label on process criteria only. Other labels (e.g. Energy City) use a mix of process and output/outcome criteria.

ISEMOA and SHIFT both conclude that some kind of certification or labeling is necessary to make the QM-scheme an attractive product. The possibility of benchmarking with other organisations is described as one success factor. SHIFT is currently considering whether to pursue a 3-level (gold, silver, bronze) labeling system or the 2-point scale used by the Swiss Energy City scheme. The label will be based on a review of cities‟ performance according to individual indicators, and an overall score for each of the categories: processes, outputs and


03/01/2012 63 TTR

outcomes. SHIFT also intends to make recommendations for improvement in each transport mode category. Participating cities receive a label which shall be valid for no longer than five years.

Of course, this overall approach of SHIFT is the most integrated way to assess sustainable mobility, but this overall approach seems to be very complicated and time consuming. This problem is also faced by MaxQ, where the auditing approach becomes more difficult, depending on the ambition and the level of development of the organisation. If SHIFT follows this approach, then the project‟s method might become attractive only for cities which have already made major efforts in assessing their quality of transport planning, and which have major resources.

Of the schemes reviewed, only BYPAD has experienced significant uptake; (approximately 100 cities have been audited at least once). The BYPAD certificate recognises the efforts of the participating municipality for improving its cycling policy through total quality management. It should be seen as a quality label for cycling policy which can be applied in any city or region that wants to work on cycling policy improvements. BYPAD has left a legacy of both a serious quality improvement tool, and a strong network of cycling cities/regions and cycling experts. BYPAD has demonstrated the relevance of having an integrated cycling policy in cities/regions. Different national and regional cycling strategies (e.g. in Austria, Germany and Czech Republic) are advising the use of BYPAD as a quality management tool to improve local cycling policy. In the Czech Republic, the awarding of subsidies is coupled with the application of BYPAD. In Nordrhein-Westfalen (Germany), the application of BYPAD can be co-financed by the state government (BYPAD 2008).

Others schemes have been of limited use. Although 2,100 cities are registered for CAF, it is not known how many of these actually use it. Incentives for cities to join are required: the Swedish Road Safety Audit was adopted widely because participating cities became eligible for additional transport funding. CIVITAS Plus cities can be awarded a CIVITAS forum prize for their specific successes in the implementation of sustainable urban mobility measures. POINTER does not provide any special certificate, but, as a result of the process of monitoring and benchmarking, cities are ranked by their achievements, according to many indicators. POINTER produces full reports on cities‟ use of the CIVITAS Plus programme, and on the implementation of measures, and provides the European Commission with an overall evaluation of CIVITAS Plus projects.

Motivating the QUEST cities will be the key, as the cities will not have a budget allocation. The auditors will have to explain in detail the QUEST methodology and the benefits of taking part. A certificate should be a strong motivation to participate. The specific context of the cities needs to be defined, but the most important principle is that there should be recognition of the efforts of the participating cities in improving their mobility policy through Total Quality Management. QUEST should grant a label to cities which actually commit themselves to climb the “ladder of development” on the basis of measures that are identified using the QUEST method. In order to safeguard the uptake of improvements, QUEST should ask the local authority involved to nominate the most promising measures among those identified in the QUEST ACTION PLAN.


03/01/2012 64 TTR

6 Conclusions

6.1 The impact of approaches to plan evaluation on the achievement of European policy goals

The key current legislative and policy statements – namely, the Action Plan on Urban Mobility (2009 - 2013), and the White Paper on the Future of EU Transport Policy, Roadmap to a Single Transport Area – are both fairly new, and it is too early to draw firm conclusions on the impact that either has had on sustainable urban mobility planning at a local level. The Action Plan has not yet been subject to a mid-term review. However, the appropriate policies are now in place, and these are supported by a number of EU funding streams. Guidance provided by the Commission is detailed, and targeted at providing practical assistance for practitioners engaged in implementing sustainable mobility action plans. Furthermore, there is a specific Action aimed at raising awareness of the funding opportunities that are available. At a more strategic level, steps have been taken to encourage interoperability and achieve a harmonisation of standards, Europe-wide, in order to facilitate the creation of modern and sustainable urban transport systems.

There are grounds for optimism for the implementation of European sustainability policies at a local and regional level, in as much as the current Action Plan was derived from a thorough and wide-ranging process of consultation with practitioners who are responsible for implementation. This means that advice and guidance contained within the Action Plan should be tailored to the needs of these practitioners. The comprehensive, practical advice given is also directly framed in the context of SUMPs. There is also a healthy emphasis on the dissemination of knowledge, through training and the showcasing of good practice, and this is supported by the availability of funding for both practitioner training and demonstration projects. This emphasis on disseminating knowledge of effective sustainable urban mobility planning is very much in keeping with the objectives of the QUEST Project, particularly with QUEST‟s goal of achieving such dissemination through the training of auditors.

A specific example of support for improving sustainable urban mobility planning is provided by the ELTIS portal, which is a networking platform through which local authority practitioners can informally benchmark, and learn from one another. Other facilities provided by the EC include the Europe 2020 Monitoring Platform and the Regio Network 2020. The Monitoring Platform enables more than 150 cities and regions to contribute to the Europe 2020 debate, whilst the Regio Network 2020 is an on-line networking resource. All of these provisions provide a further link between high-level strategies, and practitioners at a local level.

EPOMM Plus is another example of an important means for implementing EU policies in sustainable urban mobility planning. This initiative is established as the authority and networking instrument for the promotion of Mobility Management in the European Union. EPOMM Plus has a remit to concentrate on cities in the newer Member States, and on those with less experience of Mobility Management. The project is a tool that the EU can use for ensuring that sustainable urban mobility planning is incorporated into National transport policies.

A major challenge might be the prioritisation of funding for Mobility Management measures, given the current economic climate in Europe, and the accompanying austerity policies. This might be achieved through emphasising the fact that SUMPs relate, not just to environmental protection objectives, but also to the important priorities of social equity and cohesion, and economic competitiveness. There has certainly been feedback provided by practitioners that emphasising the advantages that sustainable policies have for people and the local economy, is more effective in winning support from politicians and the public, than arguments that focus purely on impacts on environmental indicators.


03/01/2012 65 TTR

Policies and strategies generally emphasise the importance of financial resources to the implementation of sustainable mobility policies. Further to this, there is a recommendation that such a resource should be secured through long-term planning of finances, to ensure that the necessary infrastructure, vehicles and equipment are available for the creation of a modern and sustainable transport system.

6.2 The pros and cons of different approaches to evaluation

As described in foregoing sections, a number of projects, identified as having an interest in the evaluation of urban mobility planning, have been investigated, for the purpose of observing what lessons can be learnt from their experiences with different evaluation approaches.

A common theme to emerge from this investigation has been the importance that many projects have placed on the setting of targets, in the early stages of planning, and the associated exercise of selecting appropriate indicators. The PILOT Project (2005-7) stated the importance of putting in place a detailed monitoring and evaluation plan, because it provides a means of anticipating potential problems that might arise later in the process. The PILOT Evaluation Manual provides detailed advice on how to set up an evaluation framework, as well as on the provision of financial support and training issues. The DISTILLATE project, which conducted a survey of UK Local Authorities‟ use of indicators in the evaluation process, concluded that its investigations supported this priority, in that the most important issue to emerge from the survey was “The use of indicators in the development of well-founded targets”. The BUSTRIP Project found that it was useful to begin the process with a simple check-list exercise, featuring a questionnaire requiring Yes/No answers to questions about different aspects of the planning process. These questions related to issues ranging from whether a working group had been set up to assist in the implementation of the plan, to whether targets had been fixed for evaluation indiators. The advantage of this was to immediately focus attention on areas that are the priority for attention.

The current CIVITAS Plus project ARCHIMEDES reports that targets set at the beginning of the process should be measurable, quantifiable, relevant and realistic. ELTIS+ suggests that targets used should be “SMART”, (i.e. Specific, Measurable, Achievable, Realistic, Time-bound), whilst the advice from MOBILIS is that targets should be realistic, but ambitious. The CIVITAS II project GUARD states that the selection of indicators in relation to such targets should proceed on the basis of relevance, completeness, availability, measurability, reliability, familiarity and independence. A further, simple piece of advice to emerge from the QUEST Project‟s survey of mobility planning professionals is that the indicators selected should be readily measurable, preferably with statistics that are currently available. It was also suggested, from this sample of professionals, that the optimum number of targets or goals to be set lies in the range from 10 to 20; this is because a larger number than this might make the data collection effort too demanding on available resources.

The ARCHIMEDES Project also advises that the evaluation process works best when it is viewed as being part of a policy making, planning and implementation cycle. This is important, as it enables the possibility for policy interventions to be modified overt time, in response to evaluation findings. The specific recommendation of ARCHIMEDES is that an evaluation report should be submitted at least every year, and several other projects reviewed mentioned an annual review of some kind, allied to a more in-depth periodical review, (which typically takes place once every four or five years). The ELTIS+ State of the Art review quotes the example of Toulouse, where a monitoring commission meets annually, with a strategic evaluation being carried out every five years.

The ARCHIMEDES Project has also identified some specific lessons learnt in relation to the engagement of stakeholders in the process. These relate to ensuring that the engagement process is inclusive, but proportional to the resources available to both the local authority and


03/01/2012 66 TTR

the stakeholders involved, and also provide advice as to methods of engagement that might be used (see sub-section 4.3.3). The MOBILIS Project advises that stakeholder involvement can be enhanced with the formation of a local “mobility group”, and that this can assist with the implementation of a SUMP. ELTIS+ also emphasises the importance of involving stakeholders early on, and at every stage of the process, including monitoring and evaluation, and provides the example of Aachen, in Germany, where a stakeholder group meets quarterly. The CO2NeuTrAlp Project describes an eight-stage co-operative Local Implementation Network.

One piece of advice from the QUEST Project‟s survey of mobility planning professionals was that it is important for the evaluation methodology selected to be appropriate to the size of the city, and that time horizons for monitoring and review should be appropriate for the time that is required for policies to be implemented. The interviews also highlighted the importance of there being adequate financial resources for evaluation to be undertaken, since respondents were aware that data collection for this activity can be an expensive and resource-intensive process. The ELTIS+ recommendation is that the budget for evaluation should be at least 5% of the total budget set aside for the SUMP process as a whole, whilst the figure quoted by the CIVITAS II Project GUARD is 10%. Respondents to the QUEST survey gave mixed views as to the value of the process of evaluation; a more specific suggestion was that emphasising the importance of the SUMP in enhancing the quality of life for people, and in bringing economic benefits, is likely to be more effective in persuading both politicians and the public as to the advantages of sustainable mobility planning than arguments relating to environmental targets.

An important recommendation of the CATCH project is that there are advantages of making links between mobility planning and land-use planning. More specifically, the project suggests formalising the link between Air Quality Plans and SUMPs (i.e. Local Transport Plans, in the UK). CATCH also highlights the importance of encouraging politicians to support and engage with the sustainability planning process, and this suggestion was supported by one respondent to the QUEST survey of mobility professionals, who stated that support from local politicians is a prerequisite for a successful SUMP.

6.3 Conclusions regarding review of projects using TQM to develop audit tools

To be able to build up a TQM system which can be used successfully it is important to be aware of success factors, problems and barriers of other QM schemes. Based on success factors which have been distilled by SHIFT and ISEMOA, recommendations for the development of the QUEST tool are summarised in section 7.


03/01/2012 67 TTR

7 Recommendations for development of QUEST audit tool

A major aim of the QUEST Project is to develop an audit tool for evaluating and improving local authorities‟ Sustainable Urban Mobility Plans. In doing this, the project intends to build upon the findings and experiences of previous projects and initiatives in the field, and also learn from the more successful aspects of existing tools. The crux of this section is to provide a summary of recommendations from the work package 3 review and appraisal process, which should be taken forward to inform the development of the QUEST audit tool, (which is the subject matter of work package 4). The review of current knowledge and approaches in relation to urban mobility management has consisted of three separate lines of enquiry; these have focused on: -

policies and legislation at a European Union level

current and recent approaches to evaluation

the experiences of other projects using a TQM approach in the development of audit tools

For consistency, recommendations emerging from the review of existing sources are presented here according to these three lines of enquiry.

7.1 Recommendations from the review of policies and legislation

SUMPs provide a framework which can help local authorities to meet the requirements of EU legislation, and so help to fulfill the wider goal of moving towards a more sustainable transport system in Europe. The purpose of reviewing such policies and legislation was to assess which might influence the design of the QUEST audit tool. The review set out to identify the most important aspects of directives and related guidelines which might be incorporated into the QUEST audit tool for the evaluation of the quality of SUMPs. These are as follows: -

the theme of integration is very important to the whole concept of a SUMP, and this includes integration of the different levels of authority (i.e. European, National, Regional and Local Government levels), and across geographical borders. A key aspect of integration is that between policy domains, so that there is coordination and liaison between transport planners and land use planners, and those responsible for energy and climate change policies. The audit tool should therefore include evidence of such coordination in its appraisal of SUMPs, and, where it is deemed to be deficient, encourage greater policy integration. For example, the tool should be used to encourage links with local Sustainable Energy Action Plans (SEAPs), and should look for evidence of awareness of land use planning policies, which might include plans for neighbourhood renewal, plans for housing projects, etc. Where there appears to be insufficient consideration of land use policies, the auditor can raise awareness of the resources such as MaxLupo, a tool designed to facilitate the integration of Mobility Management and land use planning activities, and one of a suite of such tools developed by the MAX Project. MaxSumo and MaxEva are tools specifically designed to assist local authorities with monitoring and evaluation tasks, and local authorities should be made aware of their availability, if they are not already aware. The audit should also encourage links between SUMPs and Air Quality Plans (AQPs).

another important characteristic of a SUMP is that it should address the three broad policy objectives of environmental protection, economic development and social cohesion. Certainly, there was evidence from the QUEST Project‟s interviews with mobility planning professionals that too much emphasis should not be placed on environmental objectives and indicators, since the advantages of sustainable mobility


03/01/2012 68 TTR

planning in terms of enhancing access, and the quality of life, for citizens, and the accompanying economic benefits, are seen to provide a more compelling argument for persuading politicians and the public at large of the value of SUMPs. The extent to which a SUMP considers all of these policy dimensions should be an important aspect of the audit process.

the audit tool should also take account of the extent to which the local authority engaged in sustainable urban mobility planning takes advantage of the resources, facilities and advice that are made available by the European Commission. The advantage of including this issue in the audit is that, where there is scope for an increase in the take-up of such help and advice, then the QUEST Project can provide assistance as to what is available, and how it might be obtained. An example of an important facility is the ELTIS Urban Mobility Portal, which provides guidance materials, the opportunity for the exchange of best practice, and advice on the training of Mobility Management personnel. (This portal was, however, only relaunched in March of 2011, so the main objective of including its use in the audit might be the raising of awareness about the facility).

The sustainable urban mobility planning process should be assessed in terms of whether it is comprehensive in dealing with freight, as well as passenger, issues. This is particularly important in the context of “last kilometre” deliveries in urban areas.

7.2 Recommendations from the review of approaches to evaluation

The important output from this aspect of the research is recommendations as to the essential knowledge gained, and lessons learnt, from current and previous approaches to evaluation. A priority when assessing the work of other projects in this field was to identify good practice that could be used to support cities in making progress towards improving the sustainability of their transport system. Findings from elsewhere that should be taken forward as input to the design of the QUEST audit tool are as follows: -

One of the most common areas for good practice found with evaluations from elsewhere concerns the selection of targets and indicators on which subsequent evaluations are based. The precise detail of the advice offered varies between the different guidelines, projects and initiatives, and each evaluation must achieve an appropriate balance between the use of quantitative data and more qualitative indicators, according to the local context and constraints on the availability of data. Nevertheless, the important principles that should be taken on board during the creation of the QUEST audit tool are that: -

- Precise targets and objectives, and a set of appropriate indicators that will be used to effect the evaluation, should be defined at the beginning of the planning process

- The choice of indicators should maximise the probability that the evaluation process should be effective and feasible to complete within envisaged timescales and available budgets. Several versions of formal guidance were found in the literature – including the rule of thumb that targets should be “SMART” (i.e. Specific, Measurable, Achievable, Realistic and Time-bound) – but the important principle from the point of view of auditing is that targets and indicators should be suitable in order to enable the evaluation process to be completed, given local resource and data availability constraints.

Proper involvement of stakeholders in the planning process is an issue that is widely discussed in the literature. In particular, an audit should assess the extent to which stakeholders are engaged at each stage of the process, from planning, through to implementation, and monitoring and evaluation. The stakeholders invited to take part in the process should be representative of all interested parties, and include both individuals


03/01/2012 69 TTR

and representatives of organisations. They should be adequately briefed as to the remit for their involvement, the scale of which should be appropriate to their own resources and availability.

The issue of who is involved in evaluation is an important one. Ideally, evaluation should be carried out by a body that is external to, (i.e. independent from), the main process. The audit should also consider the extent to which the evaluator(s) have the necessary skills, resources and capacity for carrying out the evaluation.

Another key element in auditing mobility plans is ascertaining the extent to which plans are in place to ensure that adequate financial resources are made available, to ensure both implementation and evaluation of the Plan. This is important, given the consensus that data collection for the purpose of evaluation can be a costly and resource-intensive activity. The auditor should be able to provide advice on funding that might be available from EU sources, such as Structural and Cohesion Funds, for larger investments, (for more sustainable vehicles and infrastructure, etc.).

An important aspect of the planning process is the extent to which local politicians are involved in this process, as this is felt to be key to ensuring the successful implementation of the Plan. Political approval for the goals and priorities of the plan can help to ensure that adequate financial resources are dedicated to both implementation of the planned policy interventions, and the necessary monitoring and evaluation activities, (see above). The audit procedure should assess the extent to which there is effective political involvement in the planning process, highlighting the extent to which political consensus and support has been achieved.

Evaluation should be part of a cyclical planning process, with a schedule in place to ensure that it is repeated at least annually. The results of evaluations should be taken into account in subsequent planning and implementation phases, before being re-evaluated on at least an annual basis.

The audit should ensure that there is evidence of the results of each round of evaluation being disseminated to the public domain, so that they might inform the public debate on sustainability issues.

The evaluation aspect of the audit should consider, as a minimum, the comprehensiveness of the plan (in terms of the range of aspects of sustainable mobility that it addresses), its efficiency in terms of its use of available resources, and its effectiveness, (in the context of delivering planned outputs).

7.3 Recommendations from the review of other projects using TQM Management

The result of the assessment in the explored schemes is an evaluation report, but the content of this report is quite different. BYPAD, MaxQ and Mediate are evaluating the status quo of the organisation via a structured method. In MaxQ this results in a prioritised shortlist of promising measures. SHIFT has the ambition to come up with recommendations for each for improvement in each of the output categories (walking&cycling, public transport, mobility mangement, optimise the land use of private motorised transport modes).

BYPAD and Mediate define a number of promising measures as a result of the self-assessment, but go further than a shortlist. Both methods develop an Action plan or Quality plan which will be set up by the auditor based on the outcome of a consensus meeting of the relevant stakeholders. The action plan makes the agree actions concrete by allocating a timetable, specific budgets and responsible persons and departments to the agreed objectives and action. This is also a strong recommendation of the Civitas Pointer project: The total quality assessment should include also personnel and staff allocation to sustainable mobility measures implementation.


03/01/2012 70 TTR

This gives good guidance for the QUEST Action Plan. This plan should consist of a clear improvement plan which is based on prioritising measures to be implemented, but there should not be an over-emphasis on merely “picking the low-hanging fruit”. This is diametrically opposed to the SHIFT approach which desires to be complete. QUEST‟s main aim is to support cities in actually making progress and this is all about making choices of measures which will actually be performed. Therefore the focus on QUEST will be the assessment process itself, where the stakeholders figure out what‟s best for the city itself. This way QUEST can assure that cities not only get to know what to do, but how to do so. The ACTION PLAN can include short term and long term improvements, but for all of these measures a clear improvement programme must be developed by the auditor to give guidance to the cities.

Detailed recommendations are presented in Table 3.

Table 3 Overview of recommendations based on success factors

Success factor Recommendations for QUEST

Labeling/certification QUEST should focus on certification of the efforts cities take to achieve improvements. The audit process must be well organised and well guided in order to achieve this.

In order to assure the implementation of the QUEST ACTION PLAN approval of the city council should be required to receive the actual certificate.

Benchmark QUEST should not focus on benchmarking. This is too time consuming and the capacity can be used better in order to support the uptake in the city itself. However, some indication of how the city performs in relation to similar cities – maybe through peer review – might be useful.

Stakeholder participation/ Motivation and clear responsibilities of stakeholders

Motivation and Participation of stakeholders is the key for the self-assessment approach of QUEST. Basically the city and their staff need to be ambitious and willing to work out and implement the QUEST-tool. In order to get measures realised key stakeholders must participate in the process. Therefore QUEST must define most relevant stakeholder-groups which need to participate in the process. The set up stakeholder group which will be guided by the external QUEST expert must agree on goals, promising measures, responsibilities and budget in order to implement the QUEST actions.

Costs QUEST must be able to demonstrate its added value to municipalities‟ activities. Therefore QUEST should focus on its persuasiveness of clearly defined measures based on the agreement of key-stakeholders.

Relationship to law/support from official organisation authority

By now QUEST can only count on support of the STEER programme. QUEST should also prove its effectiveness to national bodies so participating cities can become eligible for additional transport funding. In the future QUEST could try to become part of an acreditate body like ISO which is well known everywhere.

Continuality/regularity Continuous improvements should be monitored on a regular basis. The cities should get good knowledge on the tool, its recognition and possible use in the future. The validity of the QUEST certificate should be limited in order to assure further improvements. Follow up visits could also be required if the results of the self-assessment process are poor (based on the judgment of the auditor).


03/01/2012 71 TTR

Simplicity/user friendly/Attractive product

QUEST needs to be an attractive product which helps cities to make process in the development and implementation of a sustainable mobility policy. QUEST must find the right balance between simplicity of use vs. effectiveness and credibility. The stakeholders must be able to find the right answers to relevant questions and must be able to come up with effective measures in an appropriate time. The results of QUEST must be clear and provide what cities actually want: QUEST believes that this is a well defined improvement programme with allocations in budget, personnel and means and timing in order to implement the QUEST actions.

Well defined scope/as good as no possibilities for interpretation

QUEST must define the scope of the audit well and describe the components and elements in a clear way in order to assess the state of the art of the city and to come up with improvements suggestions. With the focus on the ACTION PLAN QUEST were only the most promising measures are worked QUEST cannot and should not focus on completeness. The worked out measures must be well defined and allocated to reasonable stakeholders.

External auditor/external validation Sound communication on the methodology, a very good training of auditors is crucial, as this has to ensure objectivity and the auditor has to set-up the cooperation at the city level. This trained expert guides the city through the self assessment process and assures its quality. QUEST must define the skills and responsibilities of the auditors clearly.


03/01/2012 72 TTR

Appendix 1

List of web-based databases consulted

COST - European Co-operation in Science and Technology

This European Science Foundation funding initiative is divided into 9 Actions, with no. 8 being Transport and Urban Development. http://www.cost.esf.org/domains_actions/tud

CIVITAS - CIty-VITAlity-Sustainability

This EU funding programme helps cities to achieve a more sustainable, clean and energy efficient urban transport system, by implementing and evaluating an integrated set of technology and policy based measures. The CIVITAS website provides links to current and recent projects funded under the programme. http://www.civitas-initiative.org/main.phtml?lan=en

IEE - Intelligent Energy Europe projects database

This EU funding programme aims to help save energy and encourage the use of renewable energy sources. http://ec.europa.eu/energy/intelligent/projects/index_en.htm

ManagEnergy

This is a technical support initiative of the IEE programme. http://www.managenergy.net/

INTERREG - Innovation and Environment Regions of Europe Sharing Solutions

This EU programme provides funding for interregional co-operation across Europe. Websites for the most recent phases of INTERREG were consulted: -

Programme IIIC http://www.interreg3c.net/sixcms/list.php?page=home_en

Programme IVC projects database http://i4c.eu/approved_projects.html

All-linx - European Community for Mobility Management

This is a web-based platform for European professionals working locally in the field of mobility management. http://www.allinx.eu/

ELTIS - European Local Transport Information Service

This is a web portal for local transport news and events, transport measures, policies and practices implemented in cities and regions across Europe. http://www.eltis.org/index.php

Europa

This is the website of the EU. http://europa.eu/index_en.htm. Relevant sub-sites were also consulted: -

the Europe 2020 Monitoring Platform, a network of local and regional authorities and online platform. It aims to assess the Europe 2020 Strategy from the point of view of EU

http://www.cost.esf.org/domains_actions/tud

http://www.civitas-initiative.org/main.phtml?lan=en

http://www.civitas-initiative.org/main.phtml?lan=en

http://ec.europa.eu/energy/intelligent/projects/index_en.htm

http://www.managenergy.net/

http://www.interreg3c.net/sixcms/list.php?page=home_en

http://i4c.eu/approved_projects.html

http://www.allinx.eu/

http://www.eltis.org/index.php

http://europa.eu/index_en.htm


03/01/2012 73 TTR

regions and cities. Europe 2020 is the EU's strategy for smart, sustainable and inclusive growth for the coming decade http://cor.europa.eu/europe2020

Regio Network 2020, an online collaboration platform for representatives of European regions and others who are interested in the European Union's regional policy. https://webgate.ec.europa.eu/regionetwork2020

EUR-Lex, which provides free access to EU legal and related public documents http://eur-lex.europa.eu/en/index.htm

the Community Research and Development Information Service (CORDIS) http://cordis.europa.eu/.

TRKC - Transport Research Knowledge Centre

This is a database of European and national transport research activities. http://www.transport-research.info/web/.


https://webgate.ec.europa.eu/regionetwork2020

http://eur-lex.europa.eu/en/index.htm

http://cordis.europa.eu/

http://www.transport-research.info/web/


03/01/2012 74 TTR

Appendix 2

Detailed inventory of relevant European legislation/policies

Key

Sources are colour-coded by theme and also categorised by relevance.

Grey = Development √ = limited relevance

Green = Environment √√ = of some relevance

Gold = Climate change (including low carbon futures) √√√ = relevant

Blue Health and Public Safety

Lilac = Transport


03/01/2012 75 TTR

Name Reference Timeframe Overall relevance rating

Renewed EU Strategy for Sustainable Development

DOC 10917/06

http://ec.europa.eu/sustainable/docs/renewed_eu_sds_en.pdf

2006 onwards, with review in 2009 (see separate inventory entry below)

√

Relevance to SUMPs This Renewed Strategy was produced following a review in 2005 of the original European Sustainable Development Strategy dating from 2001. The Renewed Strategy is an overarching Strategy for all EU policies, which sets out how the EU will meet the challenges of sustainable development. The Strategy lists seven key challenges, one of which is sustainable transport. Various actions are proposed to meet the sustainable transport challenge. These include: in line with the Thematic Strategy on the Urban Environment, local authorities should develop and implement urban transport plans and systems, taking into account EC technical guidance from 2006 and considering closer co-operation between cities and surrounding regions.

Influence at local level There is limited influence on sustainable urban mobility planning at local level. The Thematic Strategy on the Urban Environment has more relevance and is considered in a separate inventory entry.

Reviewed by Samantha Jones


Mainstreaming Sustainable Development Into EU Policies:

2009 Review of the EU Strategy for Sustainable Development

COM/2009/400

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2009:0400:FIN:EN:PDF

2009 onwards √

Relevance to SUMPs The Review confirms that EU policy progress occurred in all of the Strategy‟s seven priority areas. However, it notes that unsustainable trends persisted in several areas, e.g. energy consumption of transport continued to rise. The Review does not contain any proposals relevant to SUMPs. However, it does refer to other recent policy developments including the launch of new actions as a follow-up to the Green Paper on Towards a New Culture for Urban Mobility.

Influence at local level This Review has limited influence on sustainable urban mobility planning at local level. The Action Plan arising from the urban mobility Green Paper is reviewed in a separate inventory entry.

Reviewed by Samantha Jones, TTR





03/01/2012 76 TTR


Europe 2020/

Resource Efficient Europe

http://ec.europa.eu/europe2020/index_en.htm /

COM/2011/21

http://ec.europa.eu/resource-efficient-europe/pdf/resource_efficient_europe_en.pdf

2010-20 √√

Relevance to SUMPs Europe 2020 is the EU's strategy for smart, sustainable and inclusive growth for the coming decade. The strategy has 5 headline EU targets. One of these relates to climate change / energy for which there are several sub-targets; the most relevant is: by 2020, greenhouse gas emissions should be 20% lower than 1990 (or even 30%, if the conditions are right). These EU targets are translated into national targets in each EU state, reflecting the level of contribution each is able to make to the EU effort. National targets are set in each state‟s annual national reform programme.

The Europe 2020 strategy refers to the use of existing tools to boost growth. This includes investment via the European Regional Development Fund, European Social Fund and Cohesion Fund to support smart transport among other activities.

Europe 2020 comprises several flagship initiatives, including Resource Efficient Europe, under the sustainable growth category. This flagship initiative aims to work towards a resource efficient, low carbon economy, providing a long-term framework for policy actions including transport. The key components of this long term framework are a series of co-ordinated roadmaps. The roadmaps most relevant to SUMPs are Moving to a Low-Carbon Economy in 2050 and Roadmap to a Single Transport Area - Towards a Competitive and Resource Efficient Transport System (EC White Paper). These are considered in separate inventory entries below.

Influence at local level

The EU's Committee of the Regions is an advisory body representing regional and local authorities in Europe. It has set up a Europe 2020 Monitoring Platform of more than 150 regions and cities. This enables members to contribute to the EU debate on Europe 2020, and facilitates the exchange of good practice between local and regional policy makers. The Europe 2020 Monitoring Platform comprises an interactive website (http://cor.europa.eu/europe2020), policy-focused workshops with Europe 2020 members, experts and observers, and an annual questionnaire collecting data from members on implementation of the Europe 2020 agenda. The EC provides another online networking platform, Regio Network 2020, to help regional and local authorities to contribute to achieving Europe 2020 goals (https://webgate.ec.europa.eu/regionetwork2020/).

Local transport projects may receive support from the three Funds referred to above. For example, the European Regional Development Fund finances the Innovation and Environment Regions of Europe Sharing Solutions

http://ec.europa.eu/europe2020/index_en.htm




https://webgate.ec.europa.eu/regionetwork2020/


03/01/2012 77 TTR

Europe 2020/

Resource Efficient Europe


(INTERREG) initiative; INTERREG IVC has provided €15m funding for 7 projects related to sustainable urban transport.

See also entries on Moving to a Low-Carbon Economy in 2050 and Roadmap to a Single Transport Area - Towards a Competitive and Resource Efficient Transport System.



Energy Efficiency Plan 2011 COM(2011) 109


2011 onwards √

Relevance to SUMPs The Energy Efficiency Plan aims to secure smart, sustainable and inclusive growth, as a contribution to the development of a resource efficient economy, and the enhancement of the security of energy supply. Transport is identified as the sector having the second-largest potential for contributing to energy saving, (after buildings), and reference is made to the forthcoming White Paper on Transport.

Influence at local level No reference to SUMPs.

Reviewed by Philip Barham, TTR



03/01/2012 78 TTR


EC Thematic Strategy on the Urban Environment

COM/2005/0718

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:52005DC0718:EN:NOT

2006 onwards √√

Relevance to SUMPs The Strategy notes that Europe‟s urban areas face a number of environmental challenges including poor air quality, high levels of traffic and congestion, urban sprawl, greenhouse gas emissions and waste generation. It is stated that local authorities have a decisive role in improving the urban environment and should be encouraged to adopt more integrated approaches to managing the urban environment via long-term and strategic action plans. However, the Strategy also indicates that the urban environment needs action at EU, national and regional levels. The Strategy identities that the EU can best support Member States and local authorities by promoting best practice and encouraging effective networking between cities.

Influence at local level The Strategy proposes a number of key actions including production of guidance on sustainable urban transport plans. It is noted that SUMPs are obligatory in certain Member States and voluntary in others, and local authorities are strongly recommended to develop and implement SUMPs. The guidance is covered in the inventory of projects/initiatives that are concerned with assessment of SUMPs.






03/01/2012 79 TTR


EC Roadmap for Moving to a Low-Carbon Economy in 2050

COM/2011/112

http://ec.europa.eu/clima/documentation/roadmap/docs/com_2011_112_en.pdf

2011-50 √

Relevance to SUMPs The Roadmap sets out a plan to meet the long term target of reducing domestic emissions by 80-95% below the 1990 level by 2050. It is part of the Europe 2020 strategy/ Resource Efficient Europe flagship initiative. The Roadmap shows how different sectors - including transport - can make the transition to a low-carbon economy over the coming decades. Regarding the transport sector, the Roadmap refers to more effective use of networks/safer and more secure operation through information and communication systems, pricing schemes to tackle congestion and air pollution, infrastructure charging, improved vehicle efficiency, increased use of clean technologies and electric cars, and expansion of public transport.

This is a high level document, so is of little relevance to SUMPs. The EC is using the Roadmap as a basis for developing sector specific policy initiatives/roadmaps. These include the new White Paper on transport 2011-50, which sets out measures to enhance the sustainability of transport across Europe (see separate inventory entry for further information).

Influence at local level There is limited influence on sustainable urban mobility planning at local level, for the reasons stated in the above paragraph.





03/01/2012 80 TTR


Ambient Air Quality Directive 2008/50/EC

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:32008L0050:EN:NOT

Ongoing √

Relevance to SUMPs Directive 2008/50/EC (21st of May 2008) relates to ambient air quality and cleaner air for Europe, and has the objective

of assessing, and then preventing or reducing, the harmful effects of poor air quality on human health and the environment as a whole. Details are provided on the assessment regime to be used in relation to specific emissions (NOx, PM10, etc). There is no specific reference to SUMPs. The planning document relevant to this directive is the Air Quality Plan, which is required should air quality levels fall below a defined threshold.

Influence at local level The assessment regimes specified in this directive relate to the measurement of emissions at the level of individual agglomerations and urban zones, but there is no reference to SUMPs at this level.



White Paper – Adapting to Climate Change: Towards a European Framework for Action

COM(2009) 147


2009-12 (Phase 1 – preparing the ground for Phase 2, from 2013)

√

Relevance to SUMPs The White Paper sets out an EU-wide strategy for reducing the vulnerability to climate change of Member States. It builds on consultation launched in 2007 by the Green Paper “Adapting to Climate Change in Europe”. Four actions are emphasised: 1. Building a knowledge base; 2. Integrating adaptation into key policy areas; 3. Delivering adaptation using a combination of policy instruments; 4. Encouraging international cooperation, (although there is an emphasis on cooperation at all levels, including the EU and national, regional and local authorities). Issues of water scarcity and the impact on farming are discussed – but not urban mobility.

Influence at local level No reference to SUMPs at this level.









03/01/2012 81 TTR


Directive on Energy End-use Efficiency and Energy Services

Directive 2006/32/EC


Ongoing √

Relevance to SUMPs This Directive relates to the commitment to control demand for energy sources through improvements to the efficiency of energy use by the end-user. The main objective is to contribute to reducing primary energy consumption, in order to mitigate CO2 and other greenhouse gas emissions, so contributing to meeting Kyoto climate change commitments. The main focus of the Directive is the use of energy in the home, but it is also stated that the transport sector has an important role to play in energy efficiency and energy savings.

Influence at local level No reference to SUMPs.



White Paper – A New Strategic Approach to Health for the EU

COM(2007) 630

http://europa.eu/legislation_summaries/public_health/european_health_strategy/c11579_en.htm

2008-13 √

Relevance to SUMPs The White Paper covers the general health strategy for Europe, addressing issues related to the ageing of the population, cross-border health threats and illnesses linked to unhealthy lifestyles.

Influence at local level No reference to SUMPs in this White Paper.








03/01/2012 82 TTR


EC White Paper on transport, European Transport Policy for 2010: Time to Decide

COM/2001/370

http://ec.europa.eu/transport/strategies/doc/2001_white_paper/lb_com_2001_0370_en.pdf

2001-11, with review in 2006 (see separate inventory entry below)

√√

Relevance to SUMPs This White Paper advocates a new approach to urban transport by local authorities reconciling the modernisation of public services with rationalisation of private car use. Although the White Paper contains an action programme extending until 2010, comprising some 60 measures to be taken at EC level, no measures relate specifically to SUMPs. The element of the White Paper most relevant to SUMPs is Part 3, placing users at the heart of transport policy, and within this, sub-section IV relating to rationalising urban transport. This places emphasis on promoting good practice.

Influence at local level The main influence at local level appears to be funding of projects demonstrating sustainable integrated urban transport measures through the CIVITAS initiative. For more information on CIVITAS, see separate inventory entry on the EC Action Plan on Urban Mobility.





03/01/2012 83 TTR


Mid term review of European Transport Policy for 2010: Time to Decide, Keep Europe Moving

COM/2006/314


2006-11 √√

Relevance to SUMPs This review identified that the White Paper measures must be adapted to a new context of an enlarged Europe, rising petrol prices, Kyoto commitments and globalisation, with a policy that built on the continuity of sustainable mobility in Europe. The review noted that the EC would continue to promote research on urban mobility and called for more ambitious actions to change mobility in European urban areas. As a result, the EC launched a debate on urban transport policy in 2007 through its Green Paper, Towards a New Culture for Urban Mobility. This was subsequently translated into an Action Plan.

Influence at local level Influence at local level stems from the EC Action Plan on Urban Mobility, which is considered in a separate inventory entry.






03/01/2012 84 TTR


EC White Paper on the future of EU transport policy, Roadmap to a Single Transport Area - Towards a Competitive and Resource Efficient Transport System

COM/2011/144


2011-50 √√√

Relevance to SUMPs A new White Paper on the future of EU transport policy was published in March 2011. This follows on from the existing Transport White Paper and the low-carbon economy road map (see separate inventory entries), and from the first step in defining future transport policy, the EC communication A Sustainable Future for Transport: Towards an Integrated, Technology-Led and User-Friendly System.

The new White Paper calls for a reduction of CO2 from transport of at least 60% by 2050. It sets various targets relating to different types of journey, including those within cities. A big shift to cleaner cars and fuels, public transport and act ive travel is proposed - a 50% shift away from conventionally fuelled cars by 2030, phasing them out in cities by 2050. The White paper identifies the need for mixed urban strategies to reduce congestion and emissions, involving land-use planning, pricing schemes, efficient public transport services, and infrastructure for non-motorised modes and charging/refuelling of clean vehicles. It is stated that cities above a certain size should be encouraged to develop urban mobility plans, bringing all those elements together. These plans should be fully aligned with integrated urban development plans. The White Paper contains a set of measures to achieve its targets.


The White Paper measures most relevant to SUMP activity at local level are:

- innovations for sustainable urban mobility (smart mobility partnerships and demonstration projects) following up the CIVITAS programme, including initiatives on urban road pricing and access restriction

- introduction of procedures and financial assistance at European level for urban mobility audits and urban mobility plans, on a voluntary basis. The EC will set up a European urban mobility scoreboard based on common targets. It will also examine the possibility of a mandatory approach for cities of a certain size, according to national standards based on EU guidelines

- linking regional development and cohesion funds to cities and regions that have submitted current, independently validated urban mobility performance and sustainability audit certificates

- examining the possibility of a European support framework for progressive implementation of urban mobility plans in European cities

- developing integrated urban mobility in a possible Smart Cities Innovation Partnership - encouraging large employers to develop corporate/mobility management plans. - Since the White Paper was only recently published, no further detail is available yet on these measures.





03/01/2012 85 TTR


EC Action Plan on Urban Mobility

COM/2009/0490

http://ec.europa.eu/transport/urban/urban_mobility/action_plan_en.htm.

2009-13, with 2012 review of implementation and need for further action

√√√

Relevance to SUMPs This Action Plan follows on from the 2007 Green Paper, Towards a New Culture for Urban Mobility and the EC communication, A Sustainable Future for Transport: Towards an Integrated, Technology-Led and User-Friendly System. The Plan‟s aims include promoting integrated policies to deal with complexity of urban transport systems, optimising urban mobility to encourage effective integration between different transport networks, and sharing experience and knowledge.


The Plan proposes twenty actions to help local, regional and national authorities achieve their goals for sustainable urban mobility. Those most relevant to SUMP activity at local level are:

- Action 1 Accelerating the take-up of sustainable urban mobility plans The EC Commission has introduced an urban mobility dimension in the Covenant of Mayors, to promote an integrated approach linking energy and climate change with transport. It encourages incorporation of transport and mobility issues in the Sustainable Energy Action Plans (SEAPs) prepared by cities participating in the Covenant. This is covered in guidelines on SEAP development. As of June 2011, 5 SEAPs had been approved and a further 489 were under evaluation, covering a range of cities across Europe. Action 1 is also supported via the ELTIS urban mobility portal, relaunched in March 2011 (see Action 17 below). ELTIS is funded by the Intelligent Energy Europe programme (IEE).

- Action 14 Optimising existing funding sources Under the Transport theme of Framework Programme 7, a priority area has been dedicated to sustainable urban mobility. Within FP7, the EC is continuing to support the CIVITAS initiative which funds projects demonstrating sustainable urban mobility activities locally. Since 2002, €180m funding has been provided to 58 cities in phases, CIVITAS I, II and Plus. A new round of proposals for Plus II is currently being considered, for which the indicative budget is €18m. The EC is also maintaining support for STEER, the sub-programme under IEE dealing with the energy aspects of transport; e.g. there is an indicative budget of €12m for STEER projects in the 2011 round of IEE proposals currently being considered.

- - Action 17 Setting up an urban mobility observatory for urban transport practitioners. This is the ELTIS portal which shares information, data and statistics, monitors developments, and facilitate the exchange of best practice. The ELTIS portal provides access to a website specifically about SUMPs. Guidelines on developing and implementing SUMPs are available from this website (see separate inventory entry for further information).


http://ec.europa.eu/transport/urban/urban_mobility/action_plan_en.htm


03/01/2012 86 TTR


European Local Transport Information Service (ELTIS)

ELTIS portal: http://www.eltis.org/index.php

SUMP website: http://www.mobilityplans.eu/index.php

ELTIS has existed for over 10 years, relaunched in March 2011. SUMP website was launched in March 2011.

√√√

Relevance to SUMPs The ELTIS urban mobility portal is funded by the IEE programme. As noted above, it has been used to take forward some of the Actions in the EC Action Plan on Urban Mobility. It is worth considering ELTIS as a policy in its own right given its focus on sustainable urban mobility planning. ELTIS describes itself as Europe's main portal on urban mobility. It facilitates the exchange of information, knowledge and experience regarding urban mobility, and is aimed at individuals working in transport. ELTIS also provides access to a separate website about SUMPs.

Influence at local level ELTIS offers the following services:

- a regular round up of latest local, regional and European transport news - listing of urban transport upcoming events - case studies of successful examples of urban transport strategies and initiatives providing insight into success

factors - practical tools including guides and handbooks to support urban transport professionals in their work. - a range of other services, including e-forum, European policy reviews, information about funding sources,

transport statistics, photo library, training material and job offers. - Transport professionals can submit news items, event details, case studies, training resources and job offers for

inclusion on ELTIS.

The SUMP website provides:

- information about the benefits of SUMPs - - guidelines on the different stages of developing and implementing SUMPs, including assessing the baseline,

setting targets, developing measures, allocating responsibilities and funding, and evaluation. The guidelines also incorporate examples of good practice.


http://www.eltis.org/index.php

http://www.mobilityplans.eu/index.php


03/01/2012 87 TTR


EC Directive on the Framework for the Deployment of Intelligent Transport Systems (ITS) in the Field of Road Transport and for Interfaces With Other Modes of Transport /

EC Action Plan for the Deployment of ITS in Europe

Directive 2010/40/EU

http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:207:0001:0013:EN:PDF /

COM/2008/0886


Directive 2010-17/

Action Plan 2008-14 onwards, with a review in 2012

√

Relevance to SUMPs A new Directive was adopted in 2010 to accelerate ITS deployment across Europe. The related Action Plan outlines six priority areas for action, including continuity of traffic and freight management ITS services on

European transport corridors and in conurbations, and European ITS co-operation and co-ordination.

Influence at local level Under the priority area of continuity of traffic and freight management ITS services on European transport corridors and in conurbations, it is proposed to define an ITS framework architecture for urban transport mobility. Under the priority area of European ITS co-operation and co-ordination, it is proposed to set up an ITS collaboration platform to promote ITS initiatives in the area of urban mobility. The deadline for both actions is noted as 2010, and the Europa website indicates that the ITS collaboration platform will start after summer 2010. However, it has not been possible to find further information confirming implementation of both actions, and therefore it is difficult to determine influence on local sustainable urban mobility planning.


http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:207:0001:0013:EN:PDF







03/01/2012 88 TTR


Directive on the Promotion of Clean and Energy Efficient Road Transport Vehicles

Directive 2009/33/EC


Ongoing √

Relevance to SUMPs This Directive requires public authorities, and some operators, when procuring road transport vehicles, to take account of the lifetime energy and environmental impacts of these vehicles. Such considerations should include the energy consumption and emissions characteristics of the vehicles. It is intended that contracting authorities should, by their actions, promote and stimulate the market for clean and energy efficient public transport vehicles.

Influence at local level There is no reference to SUMPs.

Reviewed by Philip Barham





03/01/2012 89 TTR

Sources found and scanned, but not investigated furtherdue to lack of relevance: -

EPBD - Energy Performance in Buildings Directive (2010)

Concerted action on the Energy Performance in Buildings Directive

Energy Labelling Directive (2010)

Eco-Design Directive (recast) (2009)

Concerted Action on the End-use Efficiency and Energy Services Directive


03/01/2012 90 TTR

Appendix 3

Detailed inventory of projects and documents relevant to SUMP evaluation

Key

Sources are categorised by relevance. All sources are transport or energy related.

√ = limited relevance

√√ = of some relevance

√√√ = relevant


03/01/2012 91 TTR

EU CIty-VITAlity-Sustainability (CIVITAS) / Framework Programme (FP) initiatives and projects

Project/initiative/document Reference Timeframe Overall relevance rating

CIVITAS evaluation programme http://www.civitas-initiative.org/

http://www.civitas-initiative.org/docs1/CIVITAS_METEOR_Final_Cross_Site_Evaluation_Report.pdf

http://www.civitas-initiative.org/docs1/CIVITAS_GUARD_Final_Overview_of_Evaluation.pdf

http://www.civitas-initiative.org/docs1/CIVITAS_GUARD_Final_CBA_Analysis_Report.pdf

http://www.civitas-initiative.org/docs1/CIVITAS_GUARD_Final_Policy_Issues_Report.pdf

http://www.civitas-initiative.org/docs1/CIVITAS_II_Policy_Advice_Notes_06_Mobility_Management.pdf

2002-9 √√

Funding programme Partners and other stakeholders involved

EU, Framework Programme 7 CIVITAS I: METEOR consortium, co-ordinator NEA Transport Research and Training (Netherlands); plus partners from all CIVITAS projects

CIVITAS II: GUARD consortium, co-ordinator NEA Transport Research and Training (Netherlands); plus partners from all CIVITAS projects

http://www.civitas-initiative.org/















03/01/2012 92 TTR

CIVITAS evaluation programme

Remit

The CIVITAS initiative helps cities achieve a more sustainable, clean and energy efficient urban transport system by implementing and evaluating an ambitious, integrated set of technology and policy based measures. To date there have been three rounds of CIVITAS projects, I, II and Plus. Within CIVITAS I (2002-6) there were 4 projects with a total of 19 cities involved. Within CIVITAS II (2005-9) ) there were 4 projects with a total of 17 cities involved. Within CIVITAS Plus (2008-2012) there are 5 projects, with a total of 25 cities involved. Bids to a new round of Plus II projects are currently being considered.

Each phase of CIVITAS has involved a cross-project evaluation. The CIVITAS I evaluation activity was co-ordinated by the METEOR consortium. The CIVITAS II evaluation activity was co-ordinated by GUARD (Plus projects are ongoing, so evaluation results are not yet available.)

(Specific CIVITAS projects of particular relevance are reviewed below.)

Approach to SUMP evaluation Within the cross-project evaluation programmes, many different types of sustainable transport measures were evaluated, including SUMPs. Key elements of evaluation included :

- process evaluation to understand the process behind measure development and implementation. This identified barriers and drivers that affected measure development and implementation, and assessed the level of influence of each barrier/driver

- impact evaluation, including technical, social and economic impacts. Indicators were used to evaluate impacts. This involved comparing situations before and after measure implementation, and comparing the results of measure implementation with the results of a „do-nothing‟ scenario. Impact evaluation also involved cost-benefit analysis, based on a standard formula, However this was conducted only for a limited number of measures, due to issues with the quality of impact data collection and evaluation.

- assessing the transferability of measures to other locations.


03/01/2012 93 TTR

CIVITAS evaluation programme

Lessons learnt regarding SUMP evaluation

The following lessons were identified from various GUARD reports. While these were not specific to SUMP evaluation, they are pertinent to it:

- maintaining a proper balance between delivery and evaluation, in terms of resourcing, is a difficult issue which needs regular attention. Resource constraints can lead to inappropriate evaluation approaches being used. From the start of a measure, up to 10% of the budget may be needed for evaluation. An evaluation plan is required

- selection of indicators for evaluation should be based on the following main criteria: relevance, completeness, availability, measurability, reliability, familiarity and independence

- too often a focus was placed on a limited set of indicators - ongoing evaluation is necessary, starting at the very beginning of the process of measure development by

analysing the current situation including collection of existing data. Evaluation should subsequently show the lasting effects of an intervention and at what frequency this should be adjusted

- the details of many measures are developed over their duration. Such developments, or delays in implementation, can lead to changes in evaluation requirements. These may be difficult to overcome, particularly if some “before” data has already been collected. This requires careful control

- local politicians may not welcome publication of results which show “failure” of a measure.

Notes to inform recommendations section of report D3.1

We need to consider if the QUEST audit tool should utilise a standard set of indicators. If so, an appropriate range of indicators need to be covered, and the selection criteria referred to above could be taken into account.

Reporting of QUEST audit results may need careful handling with local politicians.

We may need to consider if there any elements of evaluation that are unrealistic for the QUEST audit tool to cover. For example could cost benefit analysis be too complicated? We should be clear about the scope of the audit tool, and acknowledge any elements that it does not address and explain the reasons for their omission.

Some of the lessons from CIVITAS evaluation are similar to those revealed by the CIVITAS ARCHIMEDES project - see below for notes to inform recommendations in report D.31.



03/01/2012 94 TTR


ARCHIMEDES - Achieving Real CHange with

Innovative transport MEasures Demonstrating Energy Savings

http://www.civitas-initiative.org/project_sheet?lan=en&id=9 2008-12 √√


EU, CIVITAS Plus Aalborg Kommune (Denmark); Nordjyllands Trafikselskab (Denmark); Primaria Municipiului Iasi (Romania); Regia Autonoma de Transport Public Iasi (Romania); Technical University (Romania); Ayuntamiento de Donostia-San Sebastián (Spain); Compania del Tranvia de San Sebastian Sociedad Anonima (Spain); Grupo de Estudios y Altrenativas 21, S.L. (Spain); University of the Basque Country (Spain); Instituto Vasco de Logística / Logistikako Euskal Erakundea (Spain); Brighton & Hove City Council (UK); Brighton & Hove Bus and Coach Company Limited (UK); Statutory City of Usti N.L. (Czech Republic); Comune di Monza (Italy); Project Automation S.P.A (Italy); Trasporti Pubblici Monzesi (Italy); Transport & Travel Research Ltd; Euro Project Consult

Remit ARCHIMEDES is a large-scale demonstration project addressing problems and opportunities for creating environmentally sustainable, safe and energy-efficient transport systems in medium-sized urban areas. These areas are categorised into “Leading Cities” which are transferring experience and best practice to “Learning Cities”. Project measures include development of SUMPs by the two Learning Cities, Monza (Italy) and Ústí nad Labem (Czech Republic). Leading Cities are making their expertise available to the Learning Cities through Training and Learning activities, including training workshops for stakeholders.

Approach to SUMP evaluation

City approaches to SUMP evaluation, as identified in training activities, include:

- using self assessment - identifying all relevant stakeholders (including citizens) and involving them in all stages of SUMP process,

including evaluation - defining and monitoring targets and indicators - process evaluation, a continuous activity from the outset - impact evaluation, occurring after implementation phase in order to verify achievements and policy outcomes - evaluating SUMP planning, as well as implementation of actions and budget. This includes establishing

procedures to handle discussion of detailed action programmes - producing annual progress reports.

N.B. ARCHIMEDES is not due to finish until September 2012, and SUMPs/associated evaluation arrangements for the learning Cities will be finalised until towards the end of the project. Therefore additional information about approaches to SUMP evaluation may become available at that stage. The project is graded as being of medium level relevance to QUEST based on currently available information.

http://www.civitas-initiative.org/project_sheet?lan=en&id=9


03/01/2012 95 TTR

ARCHIMEDES


Concerning stakeholder participation:

- this should be representative and inclusive, but also manageable and cost-effective - the interests, resources and capacities of different stakeholders should be taken into account - planning of main involvement activities should be concluded before initiating the SUMP process - different involvement tools and techniques should be considered and the most suitable selected according to local

context/target group - stakeholders should be provided with sufficient and transparent information regarding SUMPs, to ensure informed

involvement and confidence in the process - dealing with changes in stakeholder role can be challenging, e.g. in terms of continuity of input.

Targets should be measurable, quantifiable, relevant and realistic, in order to allow monitoring of SUMP progress and assessment of the efficiency and effectiveness of the measures taken.

A SUMP is not a closed planning tool, but a cycle of policy making, planning and implementation. Therefore, evaluation mechanisms should allow assessment and modification of efforts over time, and inform future improvements. (This links to the concept of total quality management.)

Draw on national SUMP guidance where applicable.


It should be emphasised that SUMP evaluation is an ongoing process. Ideally it needs early consideration within the process of SUMP development. This links to planning and timing of QUEST audits. Therefore cities interested in participating in QUEST audits should be given advice on when to plan for and schedule audits.

The QUEST audit tool should:

- provide a mechanism for involving stakeholders. This should be sufficiently flexible to reflect local circumstances, including the needs and capacities of different stakeholders

- provide a means of monitoring and evaluating SUMP targets and indicators - ideally incorporate both process and impact evaluation - ensure that all stages of SUMP development are evaluated.

We need to find a way of ensuring that the QUEST audit process is co-ordinated with any SUMP evaluation arrangements mentioned in national/regional SUMP guidance.

Regarding QUEST audit results, consideration should be given to arrangements for:

- reporting results, e.g. in what format, how often and to whom? - following up on audit results. How will these be used to inform future policy development?



03/01/2012 96 TTR


RENAISSANCE -Testing Innovative Strategies for Clean Urban Transport for historic European cities

http://civitas.eu/project_sheet?lan=en&id=13

2009-13 √


EU, CIVITAS Plus Perugia (Italy); Bath (UK); Gorna Oryhavitsa (Bulgaria); Skopje (Macedonia); Szczecinek (Poland); NEA Transport Research and Training (Netherlands); FIT Consulting (Italy)

Remit The RENAISSANCE project is an integrated project between 5 European cities, (Perugia (Italy) - the Co-ordinating city for the group, Bath (UK), Gorna Oryhavitsa (Bulgaria), Skopje (Macedonia) and Szczecinek (Poland).) which are regarded as historical cities of World-wide recognition. These cities face additional transport problems in comparison with other cities, which arise from seasonal tourist demand. RENAISSANCE aims to develop a valuable, reliable and integrated package of access and mobility measures to ensure cleaner and safer historic cities but, in doing so, manage the potential conflicts that arise between implementing modern innovative transport solutions and protecting the valuable historic fabric of the city.

This is relevant to SUMP from the perspective of integrating modern transport technologies in historic centres.

Approach to SUMP evaluation The approach is to provide a clear and concise evaluation of individual measures and integrated packages of measures. To evaluate both the direct and indirect impacts of the measures. To predict the potential impacts of implementing similar measures elsewhere in Europe. To measure the process of undertaking innovative measures such as those in RENAISSANCE and the key barriers and drivers that affect the success of measures.

Lessons learnt regarding SUMP evaluation Work is ongoing. It has not been possible to establish lessons learnt at this stage of the project.


The importance of developing “clean” mobility solutions for historic cities.

Reviewed by Leisa Stephenson, TTR

http://civitas.eu/project_sheet?lan=en&id=13


03/01/2012 97 TTR


MOBILIS - Mobility Initiatives for Local Sustainability

CIVITAS website area: http://www.civitas-initiative.org/project_sheet?lan=en&id=2

Final evaluation report: http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Evaluation%20Report.pdf

Policy recommendations: http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Policy%20Recommendations%20Report.pdf

Final transferability report: http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Transferability%20Report.pdf

2005-9 √√


EU, CIVITAS II Toulouse (France); Debrecen (Hungary); Ljubljana (Slovenia); Venice (Italy); Odense (Denmark); Mobiel21 (Belgium); Rupprecht Consult – Forschung & Beratung GmbH (Germany)

Remit CIVITAS MOBILIS partners cities and their local mobility stakeholders – strive to create a culture for clean urban mobility in the framework of sustainable development, ensuring involvement of all relevant stakeholders and participation of citizens.

MOBILIS implemented radical strategies for clean urban transport in all five cities, building on a broad range of policies and instruments. The project contained a total of 46 measures, which addressed all eight CIVITAS policy fields, integrated into a detailed work programme over a duration of four years.


The MOBILIS project considered that a real change towards a true comodality could only take place through a convincing set of integrated policies. This included land use, mobility planning, social policies, health policies, and alternative mobility favouring actions, each adapted to the “unique” local context.

Evaluation concepts in MOBILIS were consistent with the GUARD approach, as described in document D.2.1 Framework for Evaluation, 8th February 2006. They were developed in the evaluation plan D.3.1 V8.

In MOBILIS, for each measure, the evaluation tasks depended on the nature of the measure. They were as follows:

- evaluation of the functionalities



http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Evaluation%20Report.pdf



http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Policy%20Recommendations%20Report.pdf



http://www.civitas-initiative.org/docs/CIVITAS%20MOBILIS%20Final%20Transferability%20Report.pdf




03/01/2012 98 TTR

MOBILIS


- evaluation of the technical impacts (also called Impact Evaluation in the text). - evaluation of the acceptability - cost-benefit evaluation data - process evaluation - gender issues.

Lessons learnt regarding SUMP evaluation The use of mobility work groups enforced monitoring and management during the implementation of SUMPs.


Based on the MOBILIS experience, a focused set of policy recommendations was developed for local and European politicians and policy makers to will help each European city to realise a sustainable urban mobility culture. Recommendations for local policy makers included:

- a new urban mobility culture is an integral part of wider policy making - set realistic but ambitious objectives - engage citizens in mobility planning and implementation - learn by doing, share experience and contribute to inclusive governance in sustainable mobility planning and

implementation - the creation of local mobility workgroup fosters the realisation of a SUMP. “Almost all large and medium sized

European cities have developed a sustainable urban mobility plan. The MOBILIS experiment in Debrecen proved that the institutionalisation of a mobility workgroup bringing together all authorities with mobility competences helps to develop a strategy and to realise the action plan. An annual monitoring of the advancements will help to manage its realisation.

- state your financial investments in a SUMP - reinforce the integration of different mobility planning and management.

Recommendations for EU policy makers included:

- introduce a CIVITAS quality label. “CIVITAS should become a “quality label” for cities that adopt an integrated approach towards a more sustainable urban transport. The attribution of this quality label could be based on the presence of an integrated approach and the implementation of a multiannual sustainable mobility improvement strategy. Periodical assessments should lead to a reconfirmation of the label over time. This will enable local politicians to promote the adoption, and communicate on the need of a new urban mobility culture. It could be a type of “agenda 21” for urban mobility.”

Reviewed by Vicky Edge, TTR


03/01/2012 99 TTR


SUCCESS - Smaller Urban Communities in Civitas for Environmentally Sustainable Solutions

CIVITAS website area: http://www.civitas-initiative.org/project_sheet?lan=en&id=4

Policy recommendations: http://www.civitas-initiative.org/docs/SUCCES-513785%20Policy%20Recommendations%20_8_.pdf

Final report: http://www.civitas-initiative.org/docs/CIVITAS%20SUCCESS%20Final%20Report.pdf

Project final brochure – methodological guide: No weblink, e-mailed over to VE by Milena Perpelea ([email protected])

2005-9 √


EU, CIVITAS II La Rochelle (France); Ecole d'Ingénieurs de La Rochelle (France); Preston City Council (UK); Ploiesti (Romania); Regia Autonoma de Transport Public Ploiesti (Romania); Universitatea Petrol-Gaze Ploiesti (Romania); Lancashire County Council (UK); Preston Bus Ltd (UK); Transport and Travel Research Ltd (UK); South Ribble Borough Council (UK).

Remit La Rochelle, Preston and Ploiesti represented well the medium-sized cities in Europe, which face specific transport issues: small surface and volume area, which implies greater mixing and interconnection of activities; a lack of funding, which can be a barrier to the implementation of sophisticated technologies; a need to adopt political solutions quickly; unfamiliarity with the complexity of European projects; seasonality of transport uses. The SUCCESS cities decided to use the latest technologies of clean vehicles allied with other measures to create places where citizens could enjoy a high quality environment and travel easily and safely, to build local partnerships to tackle sustainable mobility issues, to develop efficient management systems, and to adopt a new approach to urban transport.

Approach to SUMP evaluation No relevant information given.

Lessons learnt regarding SUMP evaluation TTR and EIGSI prepared a methodological guide built on the three case study cities and containing lessons in terms of institutional, technical, economical and cultural barriers. The objective of the guide was to transfer the main findings from the project and help medium sized city authorities to implement SUMPs.

No information provided on SUMP evaluation, only on evaluation of specific measures implemented. SUCCESS noted that political decisions, technical coherence and involvement of actors were the foundations of urban mobility projects, which should be brought together so as to implement integrated projects. In SUCCESS this was done through excellent local partnerships that brought organisations together in a positive way, and through exchange of experience between the three cities both at political and technical levels.



http://www.civitas-initiative.org/docs/SUCCES-513785%20Policy%20Recommendations%20_8_.pdf

http://www.civitas-initiative.org/docs/SUCCES-513785%20Policy%20Recommendations%20_8_.pdf

http://www.civitas-initiative.org/docs/CIVITAS%20SUCCESS%20Final%20Report.pdf

http://www.civitas-initiative.org/docs/CIVITAS%20SUCCESS%20Final%20Report.pdf

http://www.civitas-success.org/e_cities_12.htm




http://www.civitas-success.org/e_cities_15.html


03/01/2012 100 TTR

SUCCESS


Bringing together all relevant stakeholders and exchange of experience.

Reviewed by Vicky Edge, TTR


GUIDEMAPS - Gaining Understanding of Improved Decision Making and Participation Strategies

http://www.transport-research.info/web/projects/project_details.cfm?id=5625&page=results

The handbook “Successful transport decision-making”:

http://www.civitas-initiative.org/docs1/GUIDEMAPSHandbook_web.pdf

2002-4 √√


EU, 5th RTD FP Austria: Universitaet für Bodenkultur, Institute for Transport Studies, Vienna

Czech Republic: CDV Centrum Dopravního Výzkumu (Transport Research Center), Brno - Magistrat mesta Brno, Utvar hlavního architekta, Brno

France: DREIF/DIT/Groupe Etudes des Stratégies des Transports, Paris

Germany: RWTH-Aachen, ISB Institut für Stadtbauwesen und Stadtverkehr (Co-ordinator), Aachen SOCIALDATA Institut für Verkehrs- und Infrastrukturforschung GmbH, Munich

Greece: Aristotle University of Thessaloniki, Transport Section at the Department of Civil Engineering

Spain: Sener Ingenieria y Sistemas, S.A., Madrid - Consorcio Regional de Transportes de Madrid, Área de Estudios y Planificación, Madrid

United Kingdom: UoW - University of Westminster, Transport Studies Group, London - PTRC - Education and Research Services Ltd, London







03/01/2012 101 TTR

GUIDEMAPS

Remit

The principle aim of GUIDEMAPS was to identify procedures and tools to improve policy decision making and achieve sustainable mobility throughout the EU, by overcoming barriers and delivering better policy outcomes.

The detailed objectives of the GUIDEMAPS project were to:

- look at the state of the art of decision making and engagement in the transport planning in Europe

- identify the barriers to successful decision making

- identify procedures and tools to improve policy decision making and achieve sustainable mobility throughout the EU

- to propose a set of guidelines and tools for overcoming barriers and to design efficient project management.

The project workplan was based on different steps:

- methodological and contextual framework, based on a broad overview of current practice in decision making and public participation processes (WP1)

- synthesis of key factors and best practices (WP2)

- a sound methodological basis for analysing and identifying barriers, as well as for managing successful decision making and participation processes (WP3)

- comprehensive, relevant and user friendly guidelines (WP4);

- field test of guidelines/handbook through selected case studies (WP5)

- dissemination of handbook and training-course activities (WP6).


The main aim of the GUIDEMAPS project was to develop a best practice guide to decision-making techniques in transport planning. Based on the extensive project research and the experiences of 20 practice examples from seven European countries in total, the main outcome of the GUIDEMAPS project is the handbook “Successful transport decision making'” The handbook gives a practical overview of good practice in stakeholder engagement, public participation and project management for local and regional transport projects. Among the tools and techniques presented are those that are already well known and well accepted, but there are also a number of tools that are relatively new for the transport sector in Europe. Included in the handbook are:

- general principles for improving transport project management and stakeholder engagement - descriptions of the most essential project management techniques - details of the most commonly used stakeholder engagement tools - clues about the costs of tools and techniques - advice about how to identify and overcome barriers.


03/01/2012 102 TTR

GUIDEMAPS


The handbook is designed to support the decision makers involved in local and regional transport planning in Europe. It is primarily aimed at transport professionals working in local authorities or transport companies, but it is also relevant to all stakeholders involved in the decision making, engagement and project management process: elected officials, community leaders, transport operators or financiers, campaign groups, NGOs and interested citizens. The handbook is not designed to be prescriptive, but rather to encourage “individual discoveries” and so enable the reader to apply appropriate project management and engagement tools to their particular situation - since there is no single “miracle recipe” for successful transport decision making. Generally, one characteristic of successful transport decision making and implementation is that it is accepted by a large number of people and by the main stakeholders. Another is that it is completed within the agreed period of time and within budget.

The section 'concepts' is the starting point for improving the transport planning process. It explains the three concepts of the GUIDEMAPS handbook (barriers, project management and engagement) and describes how these concepts interrelate. The section “tools” describes a group of related techniques - both for project management and engagement. These pages provide information which is common to the group of related techniques, such as aims and useful hints, and the barriers which may be encountered along with suggested solutions. It also shows and gives advice on which “detailed” techniques can be used.

The “fact sheets“ presented in Volume 2 describe each of the techniques in much more detail, and are referenced back to the relevant tool. The purpose of each fact sheet is to provide detailed and practical advice on planning or designing for the use of an individual technique, and on how to apply it appropriately to achieve good results.

The CD-ROM version of the GUIDEMAPS handbook contains an interactive version of both volumes of the handbook and in addition to the sections described above, the CD-ROM also contains a description of all the practice examples cities who took part in the project, and a database of further references and resources relating to project management and engagement.


03/01/2012 103 TTR

GUIDEMAPS


The lessons learnt from the survey undertaken in GUIDEMAPS indicate that there is a wide variation in the practice of decision making between and within European countries. Findings suggest that the type of project makes a big difference to the form and content of the decision making process. However, in general it appears that national factors are more important in explaining differences in the decision making processes than differences due to the nature of the project being undertaken. The results of the survey revealed that there is a strong dominance of official actors (decision makers) within local authorities in transport planning schemes. This provided an interesting focus for understanding the decision maker's needs for the development of the handbook. The experience gained from the survey indicated that there are also a wide variety of actors involved and tools used in participation and communication practices undertaken within Europe. All countries investigated provided a degree of legal regulation for participation in urban and regional transport projects, but they are sometimes very limited. It is often a voluntary decision whether to deal with a broad based consultation or just to make an announcement. Within this range, the variations between the countries can partly be considered as different “degrees of development” of participation, more or less related to the economic situation, to the development of the democratic experiences and to the emancipation of the individuals over the power of government. Other influencing factors related to the history and culture can be observed, too. The advanced technique of mediation to overcome communication barriers did not occur in any project. The most frequently used tool identified by the information providers were public announcements, indicating the minimum requirement of participation. Furthermore, public meetings, consultation documents, complaints and hearings were often legally prescribed. These tools were also used voluntarily. The level of participatory involvement and tools used within the projects analysed revealed that there are distinct differences between many countries. In some cases, interactive participation was provided early in the project process, therefore a higher priority in some countries. While in others, consultation was less important and provided later in the project process and tends to involve less interactive participatory tools.


Nothing specific.

Reviewed by Radomira Jordova, CDV


03/01/2012 104 TTR

EU Intelligent Energy Europe (IEE) projects and documents


ADVANCE - Auditing and Certification Scheme to Increase the Quality of Sustainable Urban Mobility Plans in Cities

http://eaci-projects.eu/iee/page/inc/Popup_PDF.jsp?prid=2441

The official ADVANCE Project web-site is not yet available.

2011-14

√√√


EU, IEE FGM-AMOR (Austria) (Co-ordinator); University of Piraeus (Greece); Energy Agency of Podravje (Slovakia); Traject (Belgium), Association of Urban Transition (Romania); CINESI (Spain); UNIZA (Slovakia); City of Malmö (Sweden); EUROCITIES (Belgium); CERTU (France) and City of Szczecin (Poland). In addition to Malmö and Szczecin, the cities involved in the project are Agioi-Anargyroi (Greece); Alba-Iulia (Romania); Maribor (Slovenia); Salzburg (Austria); Terrassa (Spain) and Zilina (Slovakia).

Remit To create an auditing and certification scheme for SUMPs, in order to improve the quality of SUMPs in European cities. As well as developing and applying the ADVANCE Audit Scheme, the project will train external auditors in the evaluation of SUMPs.

Approach to SUMP evaluation The approach of ADVANCE is not dissimilar to that of QUEST. The process will begin with an analysis of the status of sustainable urban mobility planning in each of eight cities, and this will be done in the context of public transport, walking, cycling, urban planning, freight transport, parking management and other car-related measures. There will then be an assessment and a prioritisation of actions, which will lead to the creation of a Local Action Plan for each city. A major performance target will be for 80% of local stakeholders to feel that the process has improved the quality of sustainable urban mobility planning, as measured by a questionnaire survey at local stakeholder workshops.

Lessons learnt regarding SUMP evaluation It is too early to have learnt anything from ADVANCE, but QUEST clearly needs to keep in close touch with the project, given the obvious similarities.


See above comment.

Reviewed by Phil Barham, TTR

http://eaci-projects.eu/iee/page/inc/Popup_PDF.jsp?prid=2441


03/01/2012 105 TTR


EPOMM PLUS - European Platform on Mobility Management PLUS

http://www.epomm.eu/index.phtml?Main_ID=2801

2009-2012 √√√


EU, IEE STEER Programme. 11 member states (Belgium, Bulgaria, Czech Republic, Estonia, Greece, Hungary, Italy, Lithuania, Poland, Romania, SIovenia).

Remit

EPOMM is a network of governments in European countries that are engaged in Mobility Management (MM). EPOMM-PLUS builds on EPOMM and focuses on the new Member States and the states with little MM experience. EPOMM PLUS will help in that MM will be integrated in national transport policies by transferring good practices and supporting local pioneers by developing national networks Currently it consists of 11 member states (BE, BG, CZ, EE, GR, HU, IT, LT, PL, RO, SI). Each country participating has established a National Focal Point (NFP), which represents the Platform in that country or region. The NFPs link EPOMM with the national network of all possible actors in the field of MM.

The overall objective of the EPOMM PLUS project is to achieve a leap in the implementation of MM in Europe, by establishing EPOMM as the authority and the networking instrument for the promotion of MM across the EU.

EPOMM PLUS will achieve this especially by:

- transferring existing good practice (often developed with EU funding) and - supporting existing local pioneers by developing national networks of MM.

WP3 - National Networks. The main aim is to build up national platforms for MM. Therefore, EPOMM-PLUS has partners (so called Network Initiators - NI) in 22 European states

WP4 - Translation and National Event Promotion. Increases the dissemination of MM best practice on a national level by translation of existing EPOMM materials, modification of the website as well as making and disseminating the e-updates. It also promotes the National MM Workshops.

WP5 - Trainings. Develops training initiatives: train the trainer meetings, a training quality label as well as a training and trainer database - in cooperation with national and EU project developments - in order to support the dissemination of MM.

http://www.epomm.eu/index.phtml?Main_ID=2801


03/01/2012 106 TTR

EPOMM PLUS

Remit

WP6 - Policy and Legislation. Analysis of MM policy and legislation in 27 European states - through the Mobility Management Monitors (MMM) - eg. the MMM of Belgium 2009; transfer roadmaps of MM between the EPOMM member states; raised awareness within the EU of MM as a part of mainstream policy.

WP7 - Monitoring of MM projects in EPOMM-PLUS countries via MaxTools as well as in MaxEva.

Approach to SUMP evaluation WP7 - Monitoring of MM projects in EPOMM-PLUS countries via MaxTools as well as in MaxEva. NB the QUEST project has reviewed Max projects and tools in more detail within task 3.3.

The MaxExplorer is an interactive decision support guide for MM projects. It can assist decision makers and Mobility Management practitioners to select MM measures which were rated by Max project experts as the most appropriate for specific situations.

The Max project developed a quality management approach as a powerful tool to improve MM in general, but especially for cities. It helps to provide services in an organised and consistent manner and to continuously improve them based on user satisfaction and desires. The Quality Management System for MM () focuses on developing, monitoring, assessing and improving both the overall MM policy and separate MM measures. The scheme is a process, which can be adopted by any organisation for managing their mobility policy and measures. That process focuses on developing, monitoring, assessing and improving both the overall

MM policy and separate MM measures. It involves four steps (policy, strategy, implementation and monitoring and evaluation) and twelve sub steps, which are structured in a quality circle. http://www.epomm.eu/docs/mmtools/brochure/Max_Brochure_EN.pdf


Close working between cities, cooperation and exchange of information. Collection of case studies. Use of Quality MM tools.

Quality in MM:MAX Q: using Quality Management tools for :

- introducing quality management into mobility operations - achieving quality while adopting a quality culture in services - improving performance and credibility of Mobility Management processes.

Planning, monitoring and evaluating in MM: use of MaxSUMO and MaxEva. MaxSUMO: MaxSUMO aims to standardise evaluation at the European level and helps in planning, monitoring and evaluating MM projects. It provides standardised guidance during all steps of an MM project, e.g. when setting targets, defining target groups, selecting services and mobility options.

MaxEva: the aim of MaxEva database is to build up knowledge. MaxEva is available online and will stores Cities evaluation data. The more that MaxEva is used, the more MM results there will be to compare and use in planning new projects. MaxEva is an interactive web tool that guides you in the evaluation and collection of monitoring data MM

http://www.epomm.eu/docs/mmtools/brochure/Max_Brochure_EN.pdf

http://www.epomm.eu/index.phtml?Main_ID=2174&ID1=2259&id=2259


03/01/2012 107 TTR

EPOMM PLUS


interventions. MaxEva gives you an overview of results at any time during your project based on the entered data. If given the necessary data it even calculates the effects from your specific project in terms of reduced mileage and carbon dioxide. MaxEva also provide benchmarking between different projects. See MaxEva Manual (September 2010): http://www.epomm.eu/docs/1059/MaxEva_Manual_september_2010_2.pdf

Understand behaviour: MaxSem is a psychological model which explains the underlying processes involved in changing the behaviours of car drivers to more sustainable transport modes and allows you to gain a “fuller picture” of the effects of MM and to segment your target group according to the models assumptions.


Use of Quality MM tools and compilation of evaluation data for the exchange of information between cities.

For anyone carrying out evaluation of MM projects it is helpful to understand the underlying processes necessary for behavioural change to occur, and to use this knowledge in both the design and evaluation of MM projects. For practitioners it is of primary concern to know and to be able to show that the effort and the costs are justified. The EPOMM PLUS via the Max-project suite of tools helps the user in different ways to effectively plan, monitor and evaluate MM measures and projects. Change of behaviour often needs time and occurs in several steps or stages. For practical use, stage-diagnostic questions have been developed that can be used to show stage movements as one of the effects of a project, in addition to measuring overt behaviour change.


http://www.epomm.eu/docs/1059/MaxEva_Manual_september_2010_2.pdf


03/01/2012 108 TTR


MOVE Final report:

http://www.move-project.org/uploads/media/2_MOVE_Resultoriented_Report.pdf

2006-8 √√


IEE

Local Agency for Energy of the Province of Chieti (Italy; Energy Agency of Plovdiv (Bulgaria); Energy Centre Bratislava (Slovakia); The Energy Agency for Southeast Sweden (Sweden); The Basque Energy Board (Spain) ; Tipperary Energy Agency (Ireland); Klimaschutzagentur Region Hannover (Germany).

For a list of supporting Institutions, please see: http://www.move-project.org/50.0.html.

Remit

The purpose of the project was to select tools, establish methods and standards within a partnership based on existing European initiatives, that could be used to achieve changes in attitudes and behaviour leading to a change towards less energy intensive transport modes. The main objective was to encourage more European communities to work with mobility issues and to offer alternatives to the use of fossil fuels leading to a reduction in greenhouse gas emissions. These objectives were to be achieved by:

- providing methods that would help communities in their work towards less energy intensive transport modes - providing good examples and success stories that would encourage more actors to work with mobility issues - showing the reductions in greenhouse gas emissions from single projects in different countries using the tools

developed during the project - showing targets that were possible to be reached - showing the cost and health effects that would follow as a result.

One of the first steps of the project was the creation of an Energy Balance. This provided facts and figures including the energy consumption and emissions data related to transport in the area.

Secondly, the Energy Agencies were also required to set up a Virtual Mobility Agency within their agencies and set up a steering group that would help to drive the local projects.

Finally, when implementing the local projects, the SUMO methodology was used (System for EvalUation of Mobility PrOjects).



http://www.move-project.org/50.0.html


03/01/2012 109 TTR

MOVE


Based on these three aspects of the project, it was necessary to evaluate:

- the Approach to the Energy Balance ( standard data v software package) - the Organisation and Impact of each Virtual Mobility Agency.

SUMO was the process that was followed in order to evaluate the MOVE project at every stage of the project. This was a new methodology used during the course of the project so there was a lot of learning and re- learning as each of the local projects progressed through the implementation stages.

Examples of SUMO methodology and Target Process Flow Chart can be found at;

http://www.move-project.org/uploads/media/KSA_MOVE_EVALUATION_WALKING_BUS_engl.pdf

http://www.move-project.org/fileadmin/_move_/move%20docs/Final_2008-07-02_Smart_traveller_Template_Final.pdf.

Lessons learnt regarding SUMP evaluation Although the process was described as time consuming and difficult to comprehend, as the partners worked through the methodology the majority found the process useful. They found that it gave clarity and direction to the project and was a logical process. Because SUMO forces the user to evaluate every step of the process from start to finish, it was, it is reported, very easy to notice if the local project was on-track or off-track.

It was a useful methodology to use in a group project because it allowed partners to share experiences, to benchmark experiences at various levels and to know whether they were on course or off course in achieving the goals set out in the project. One of the surprises or the unexpected outcomes of following the SUMO methodology

was the large time resource required in carrying out surveys at the various levels.


SUMO methodology and the corresponding target flow charts appear to be a useful tool for systematic evaluation that can be monitored throughout the lifetime of the evaluation process and not just a report at the end of the process.


http://www.move-project.org/uploads/media/KSA_MOVE_EVALUATION_WALKING_BUS_engl.pdf

http://www.move-project.org/fileadmin/_move_/move%20docs/Final_2008-07-02_Smart_traveller_Template_Final.pdf


03/01/2012 110 TTR

Project/initiative/document Reference Publication date/ Timeframe

Overall relevance rating

ELTIS+ Guidelines: Developing and Implementing a Sustainable Urban Mobility Plan

http://www.mobilityplans.eu/docs/SUMP_guidelines_web.pdf Publication date: July 2011

Project duration: 2010-13

√√√

Funding programme Author

IEE Rupprecht Consult

Remit This is both a stand-alone working document, and an Annex to the ELTIS+ Project‟s “State of the Art of Sustainable Urban Mobility Plans in Europe”. The purpose of the document is to provide step-by-step guidance on how to prepare a SUMP, and to support the call of the European Commission‟s Action Plan on Urban Mobility for an increase in the take up of sustainable urban mobility planning in Europe. The guidelines are levelled at urban transport and mobility practitioners, but also other stakeholders who might be involved in the SUMP process.

Approach to SUMP evaluation An in-depth description is given of an eleven-step procedure for creating and implementing a SUMP. These eleven steps are, in turn, distributed among four phases, from “Preparing Well”, to “Implementing the Plan”. There are hints at evaluation and assessment in steps “5.2 Develop SMART (i.e. Specific, Measurable, Achievable, Realistic, Time-bound) targets”, “10.3 Check progress towards achieving the objectives” and “11.2 Review achievements – understand success and failure”. Step 8 is “Build monitoring and assessment into the plan”, including step “8.1 Arrange for monitoring and evaluation”.

It is noted that:

- responsibilities for conducting evaluation should be defined, along with a workplan. Ideally evaluation should be undertaken by an independent body

- a monitoring and evaluation mechanism helps anticipate barriers and drivers in preparing and implementing SUMPs. The mechanism should be defined early and become an integrated part of the SUMP document

- it should be determined how achievement of measure implementation and targets will be assessed - indicator selection (both quantitative and qualitative) for evaluation should be linked to setting SMART targets. A

data audit should be carried out to determine what data are available and what gaps exist before collecting indicator data

- evaluation should address the quality of the planning process, as well as outputs (actions taken) and outcomes (impacts of actions). A cost-benefit assessment may also be undertaken, although this is not seen as essential

- reporting on evaluation should ensure that the results feed back into a public debate, enabling all stakeholders to make modifications to the SUMP if necessary.

http://www.mobilityplans.eu/docs/SUMP_guidelines_web.pdf


03/01/2012 111 TTR

ELTIS+ Guidelines: Developing and Implementing a Sustainable Urban Mobility Plan


There is an emphasis on the importance of setting SMART targets, for later monitoring and evaluation. Examples of indicators that can be used to inform evaluation are given; these include both output indicators (e.g. provision of x km of additional cycle lanes), and outcome indicators (e.g. consequent improvements to Quality of Life (e.g. reduction in congestion).


SMART targets should be set early on in the SUMP process, as preparation for subsequent evaluation work.

The budget set aside for monitoring and evaluation activities should be at least 5% of the total SUMP budget.

It is important to plan for stakeholder involvement, and maybe some peer review from similar cities, in the monitoring and evaluation process.



03/01/2012 112 TTR

Project/initiative/document Reference Publication date/ Timeframe

Overall relevance rating

ELTIS+

State of the Art of Sustainable Urban Mobility Plans in Europe

http://mobilityplans.eu/docs/file/eltisplus_state-of-the-art_of_sumps_in_europe_july2011.pdf

Publication date: July 2011

Project duration: 2010 -13

√√

Funding programme Authors

IEE Rupprecht Consult and Edinburgh Napier University

Remit The document is a deliverable of the ELTIS+ Project, and serves as a reference manual for urban mobility professionals. It describes approaches to SUMPs in the 27 European Union states, plus four countries which contribute to the IEE Programme (Croatia, Iceland, Liechtenstein and Norway), and compares both levels of awareness and training needs. The document sets out the requirements for producing a good SUMP.

Approach to SUMP evaluation The document does not specifically focus on assessment and evaluation of SUMPs, except in the sense of describing how SUMPs should be created and implemented. For example, emphasis is placed on the fact that evaluation of SUMPs should be carried out with a participatory approach, (as should the other stages of SUMP development), in as much as all relevant stakeholders should be involved. Reference is also made to the fact that evaluation should be based on targets and performance indicators that are both achievable and measurable.

Lessons learnt regarding SUMP evaluation As previously noted, there is little in this document about the evaluation of SUMPs, but the document does provide some important and up to date background information.


Nothing of note, from this reference.





03/01/2012 113 TTR

EU Environment Programme projects and documents


PILOT - Planning Integrated Local Transport http://www.pilot-transport.org/

State of the Art Review: http://www.pilot-transport.org/index.php?id=18

Guidance Manual:http://www.pilot-transport.org/fileadmin/WP2/Pilot_EN_WEB.pdf

Sustainable Urban Transport Planning Policy Recommendations to the European Union and Member States: http://www.pilot- transport.org/fileadmin/WP2/PILOT_policy_recommendations_.pdf

2005-7 √√√


EU, DG Environment POLIS network; Rupprecht Consult; VECTRIS; Mobiel 21; Transport & Travel Research Ltd; Sylwia Klatka and Eurocities. Guidance Manual: Marc Wolfram, Sebastian Bührmann of Rupprecht Consult.

Remit The aim of PILOT was to demonstrate the preparation of SUMPs in four European cities: Braila, Evora, Lancaster and Tallinn. Building upon the experience of these four cities and relying on experts from leading local authorities and organisations in this field, PILOT proposed tools, guidelines and recommendations for the elaboration of SUMPs in other European regions and local authorities. PILOT also contributed to the work leading to the formulation and implementation of future thematic strategy on the Urban Environment.

This is highly relevant to SUMPs.


Firstly, an Expert Working Group (EWG) was convened which aimed to provide a balanced stakeholder feedback on what a potential EC Directive to provide a framework for SUMP regulation in all Member States should contain. The Group made a number of recommendations and included a description of the minimum requirements for SUMPs, and a framework for defining good practice. EWG recommendations on minimum requirement for SUMPs were:

- establish a common evaluation framework - provision of financial support - guidance - training and dissemination (of best practice) - co-ordination of current policies.

http://www.pilot-transport.org/

http://www.pilot-transport.org/index.php?id=18

http://www.pilot-transport.org/fileadmin/WP2/Pilot_EN_WEB.pdf

http://www.pilot-transport.org/fileadmin/WP2/Pilot_EN_WEB.pdf


03/01/2012 114 TTR

PILOT


In order to develop SUMPs as a tool the EWG further considered what component parts or in some cases who was ideally needed to deliver SUMPs. This became what is referred to as the framework for defining good practice:

- citizen participation - stakeholder consultation - actor co-operation - policy co-ordination - achieving integration with comprehensive planning - land use planning.

Secondly, a key source of information that informed the development of the national review framework and checklist, and informed the study as a whole was a literature review. “The PILOT project forms a key contribution to the formulation of the Thematic Strategy on the Urban Environment as set out in the 6th Environmental Action Programme. The project responds to the recommendations of the Working Group on Sustainable Urban Transport, which identified the preparation of Sustainable Urban Transport Plans (SUTPs) as an important cornerstone for promoting sustainable urban transport. PILOT particularly recognises the necessity to provide guidance for the practical preparation of SUTPs. Furthermore it sets out to develop a common indicator-based monitoring and assessment framework for SUTPs and to disseminate „good practice‟ and raise awareness about sustainable urban mobility issues. In this way, the project aims to prov ide substance to the design of the Thematic Strategy by addressing a number of strategic and operational objectives. PILOT is a research project that will demonstrate the preparation of SUTPs in four European cities (Evora, Lancaster, Tallinn, Braila) and establish a “pilot-network” for this innovative undertaking”.(Taken from D1.0 STATE OF THE ART REVIEW- SUTP DEVELOPMENT IN EUROPE).

In the final guidance document it is acknowledged that in order to enable local policy learning, both the SUMP process and the implementation of the action and budget plan need to be assessed by an independent body. This implies a monitoring of the different planning steps (identified in the Guidance document) as well as measuring policy outcomes and impacts. It is anticipated that the results of the evaluation should then feed back into the process regularly, either to improve the planning and deliberation procedures or to review the measures adopted.


Checklist and milestones

Success criteria level

- Well balanced selection of indicators - Clear responsibilities of well skilled staff members for monitoring and evaluation arrangements - Clearly defined budget and workplan for monitoring and evaluation arrangements - Focus of ex-post evaluation on outcome indicators - Clear assessment of costs and benefits of SUMPs (as far as possible).


03/01/2012 115 TTR

PILOT


Sequence of milestones to be achieved

Selection of suitable indicators finalised Workplan and responsibilities for data collection and management agreed Ex-ante evaluation finalised (scenario development, action and budget plan) Ex-post evaluation of plan implementation and impact finalised Ex-post evaluation of plan preparation finalised Evaluation reports finalised.

Stakeholder involvement in monitoring and evaluation (Involve peers from other cities for feedback process and a co-ordination on regional indicators)


Specifically, the guidance document provides a comprehensive approach to defining, setting up and running the SUMP process. It does not provide a clear method of evaluating the SUMP though.



03/01/2012 116 TTR


CATCH- Clean Accessible Transport for Community Health

http://www.cleanaccessibletransport.com/index.html

Best Practice Guide for the Clean Operation of Buses: http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/CleanBusGuide.pdf

Technical Evaluation Report:

http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/Evaluation_Report.pdf

? - 2005

√√


EU, LIFE-Environment Programme

EU, Sixth Environment Action Programme

Merseytravel (UK); Liverpool City Council (UK); Arriva North West and Wales Ltd (UK); CTP (Italy); Suceava Municipality (Romania); Transport & Travel Research Ltd (UK)

Remit CATCH was a demonstration project. It supported the EC's Sixth Environmental Action Programme by promoting sustainable mobility in order to improve air quality. CATCH was implemented in Liverpool (UK), Suceava (Romania) and Potenza (Italy).CATCH adopted an innovative and multi-disciplinary approach to reducing air pollution in Liverpool (UK), driving forward and demonstrating both hard measures, such as the provision of clean buses and new bus services, and soft measures, such as personal travel plans, calorie mapping and the production of new transport policies.

The CATCH project contributed specifically toward the design phase of some of the walking and cycling initiatives contained within Liverpool‟s City Centre Movement Strategy. Walking and cycling are being continually improved around the city. Specific initiatives to which CATCH contributed are; wider street crossings, on-road cycle routes and phased pedestrian priority signals. Better signage and enhanced pedestrian areas are all part of the package of works that were implemented to encourage active travel and thereby reduce air pollution.

The success of CATCH was evaluated through advanced pollution monitoring techniques and the impact on community health was evaluated by using a Health Impact Assessment (HIA) methodology. Related schemes were also implemented in Romania (Suceava) and Italy (Potenza) and the results were used for best practice guidance throughout Europe.

The results from CATCH have been reported in three separate documents; a non technical summary of key outcomes, a best practice guide and a technical evaluation of the whole project.(see reference above)

This is relevant to SUMP in terms of utilising Health Impact Assessment (HIA) methodology to evaluate the effectiveness of each demonstration. The CATCH project was not anti-car, but was about offering choices that were the cleanest available option.

http://www.cleanaccessibletransport.com/index.html

http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/CleanBusGuide.pdf

http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/CleanBusGuide.pdf




03/01/2012 117 TTR

CATCH


Health Impact Assessment (HIA) methodology. )( Inc. Community Mobility Plans)

Introduction of an Air Quality Action Plan. (At the time, Liverpool city was being designed to accommodate freer flowing public transport, less vehicular traffic on certain routes and promotion of healthier forms of transport.) See Technical Evaluation Report for more information.

Lessons learnt regarding SUMP evaluation Measure of CO2 and other air pollutants in Liverpool did not show a huge difference in fact there were higher after the project. However, it does present good practice in terms of promoting the cleanest transport options.

Feedback from cities involved in CATCH has influenced at least one UK council, working with a private sector bus operator, to install particulate traps on buses in their fleet.


- Value and role of Air Quality Action Plan - Value and role of conducting Health Impact Assessments (HIA)( including Community Mobility Plans).

Recommendations of the project included:

- develop supplementary planning documents - clarify the policy guidance linking transport and land use planning - adopt a partnership-based approach to transport-related environmental policy - encourage politicians to lead the process by providing support for a vision which the project can work towards - produce long term funding plans for transport - formalise the link between Air Quality Action Plans and Local Transport Plans - develop a formal industry standard for the presentation of comparative emissions.

http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/CATCH%20Laymans%20report%20Content.pdf (page 9).


http://www.cleanaccessibletransport.com/Revised/Reports%20for%20publication/CATCH%20Laymans%20report%20Content.pdf


03/01/2012 118 TTR


PROSPECTS - Procedures for Recommending Optimal Sustainable Planning of European City Transport Systems)

http://www.ivv.tuwien.ac.at/forschung/projekte/international-projects/prospects-2000.html

2000-2

√√


EU, DG for Energy, Environment and Sustainable Development. This project was part of the Land Use and Transportation Research Cluster.

Institute for Transport Studies (University of Leeds, UK); KTH (Stockholm, Sweden); Institute of Transport Economics (Oslo, Norway); Institute of Transport Planning and Traffic Engineering (Technical University of Vienna, Austria); VTT (Finland); University Polytechnic of Madrid (Spain); David Simmonds Consultancy (Cambridge, UK); MVA (UK).

Remit To provide cities with advice and guidance on optimal land use and transport strategies for achieving sustainability goals. The objectives, indicators and decision-making requirements of six core cities were reviewed in depth, and this review was supported with a subsequent survey of 100 cities. Outputs from the project included a Decision-Maker‟s Guidebook and supporting Methodological and Policy Guides.

Approach to SUMP evaluation The project assessed and enhanced evaluation tools for supporting the provision of assistance with decision making processes. A review of evaluation procedures and optimisation methods was carried out for this purpose. One deliverable of PROSPECTS is a 150-page report on Evaluation methods in relation to sustainability. There is no specific mention of SUMPs, but the work of the project is clearly relevant to the evaluation of sustainability policies and measures.

Lessons learnt regarding SUMP evaluation The deliverable includes lists of indicators which may be used for assessing the sustainability of urban land use and transport strategies. These indicators are listed under various categories, including Economic Efficiency, Increased Freedom of Movement for Vulnerable Road Users, Energy Use & Climate Change, Protection of Heritage Sites, Urban Sprawl, Noise, Equity & Social Inclusion, Traffic Accident Reduction, etc. as well as existing as a digest of indicators, the deliverable includes a discussion of the indicators that are most likely to be used in planning exercises. There are five such indicators in the category of Environmental Indicators.


This is an important source in terms of it providing an over-arching background reference on the evaluation of sustainability policies. It should also be consulted to inform the selection of indicators in WP4.





03/01/2012 119 TTR

Project/initiative/document Reference Publication date Overall relevance rating

Expert Working Group on Sustainable Urban Transport Plans (Final Report)

http://ec.europa.eu/environment/urban/pdf/final_report050128.pdf December 2004

√√√


EC, DG Environment, linked to the Thematic Strategy on the Urban Environment.

Rupprecht Consult

Remit The Expert Working Group on SUTPs was set up to support preparation for the Thematic Strategy on the Urban Environment. This document reports on the findings of this Expert Group. The Group‟s remit was to take a view regarding the specific design of procedures, instruments and measures needed for SUTPs, the contents of a potential EC directive, and the support actions necessary for enhancing SUTPs. Recommendations for devising an EC Directive are weighted according to whether they constitute Obligations or Advice. It is the obligatory recommendations that constitute the minimum requirements for an SUTP.

Approach to SUMP evaluation The Expert Group defines a five-step implementation mechanism for SUTPs, which includes “Monitoring and Evaluation” as a fifth step. One obligatory recommendation is that engagement with stakeholders must be built into every stage of the process, including that of evaluation, and that these stakeholders must be representative of all sections of society. (Gender equity and equality is given particular emphasis). Also obligatory is the need for the SUTP implementation to be monitored according to a set of indicators defined early on in the implementation process. It is suggested that regular progress reports should be prepared at regular intervals (e.g. annually), and that these reports should be published widely. The Group specifies that the evaluation process should consider both the quality of the SUTP, and the quality of the implementation process. It is also recommended that periodic “sanity checks” should be carried out, with the input of stakeholders, members of the public and peers from other cities.

Lessons learnt regarding SUMP evaluation The document provides a detailed description of what the EC should include as being obligatory and advisory in a directive that is designed to encourage the widespread creation and implementation of SUMPs. A detailed recommendation is that the EC should define a compulsory set of key indicators to be used, and that these indicators should be rigorously defined, in order to ensure comparability. Reference is made to several large sets of indicators already available, as an example of the indicators that might be used. It is also recommended that the EC should define minimum targets for these indicators, based on existing EC environment and sustainability legislation – these defined minima should form the basis of targets set at a national level, which should be set by Member States, who should also be responsible for monitoring performance against these.

http://ec.europa.eu/environment/urban/pdf/final_report050128.pdf


03/01/2012 120 TTR

Expert Working Group on Sustainable Urban Transport Plans (Final Report)


The report identifies France, Italy and the UK as being the most advanced European countries in terms of national regulations for local transport planning, and so makes a comparative analysis of the situation in the three countries.



03/01/2012 121 TTR


Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions, (Final Report)

http://ec.europa.eu/environment/urban/pdf/sutp.pdf October 2005

√√√

Funding programme Authors

European Commission DG Environment, Unit D4 – Health and Urban Areas

Rupprecht Consult and Trasporti e Territorio

Remit The report was commissioned in recognition of the contribution that the transport sector makes to emissions of NOx, PM, VOC, O3, CO2 and noise in cities. The document provides a review of current empirical evidence for the impact of SUMPs on health conditions in cities, and overall quality of life, and identifies current knowledge gaps.

Approach to SUMP evaluation This document evaluates SUMPs from the point of view of their impact on NOx, PM, VOC, O3, CO2 and noise, according to available evidence. In practice, the report covers over 50 specific assessments of policies which target transport emission reduction. The four broad categories of policy featured are policies which reduce the need for motorised transport, encourage modal shift to more sustainable means, develop clean and quiet means of transport and improve transport efficiency. A major aim of the study is to inform policy makers as to the extent to which there are complementarities between the various studies. The study also specifically considered different policy modelling approaches that were available. The approach of the study was to prioritise assessments which provide a quantitative assessment of policy impacts (on emissions and noise, etc.), and qualitative advice on the creation of integrated transport policy packages to maximise impacts and thus sustainability.

Lessons learnt regarding SUMP evaluation Policies and measures are categorised and dealt with in four categories: 1. Reduction of the need for motorised transport; 2. Encouragement of modal shift; 3. Development of cleaner, quieter transport systems; 4. Improvements to transport efficiency. It is suggested these are the fundamental policy categories that should be used as a basis for policy design, (so it would probably be advisable to follow this suggestion). The study finds that there are numerous synergies between the policies that make up SUMPs, and also identifies some adverse impacts. These synergies and impacts are examined in detail, in what the report describes as an analysis of “SUTP efficiency”. The usefulness of the report in terms of examining the general efficiency and effectiveness of SUMPs is limited by the fact that most of the assessments covered concentrate on the issue of a fairly narrow range of emissions: NO2, PM10 and CO2.

http://ec.europa.eu/environment/urban/pdf/sutp.pdf


03/01/2012 122 TTR

Sustainable Urban Transport Plans and Urban Environment: Policies, Effects and Simulations. Review of European References regarding Noise, Air Quality and CO2 Emissions, (Final Report)


Should adopt the 4-way categorisation of sustainable transport policies.

When assessing the impact of SUMPs, take all types of impact into consideration, (not just the “headline emissions”).



03/01/2012 123 TTR


Sustainable Urban Transport Plans: Preparatory Document in relation to the follow-up of the Thematic Strategy on the Urban Environment. (Technical Report – 2007/-018)

http://ec.europa.eu/environment/urban/pdf/transport/2007_sutp_prepdoc.pdf

September 2007 √


EU, DG Environment Not specified.

Remit This is a summary of best practice in sustainable urban mobility best practice. It aimed to assist the consultation on the Green Paper on Urban Mobility. The Annex to this document summarises the good practice examples, and sources of practical information, that are available. This Annex provides links to high-level initiatives, such as CIVITAS, ELTIS and EPOMM.

Approach to SUMP evaluation The document describes the state of the art of sustainable urban mobility on the basis of stakeholder consultation, research carried out in land use and urban transport, and a report on the EC‟s Working Group on SUTPs. There is no mention of monitoring or evaluation.

Lessons learnt regarding SUMP evaluation There is no mention of monitoring or evaluation.


See above comment.





03/01/2012 124 TTR

EU regional funding projects


BUSTRIP - Baltic Urban Sustainable Transport Planning and Implementation

http://www.bustrip-project.net

2005-7 √√√


EU, Innovation and Environment Regions of Europe Sharing Solutions (INTERREG) IIID Baltic sea region

Union of Baltic Cities; Chalmers University of Technology; Bremen; Gdynia; Göteborg, Kaunas; Kouvola region, Liepaja; Pernu; Sundsvall; Tartu; Turku; Vilnius; Örebro.

Remit Supporting partner cities in developing transport planning processes. Preparing new urban sustainable transport plans and revising existing plans.

Approach to SUMP evaluation Evaluation by self assessment and peer review process. The results of the peer review should assist the cities to prepare or improve their SUMP.

Lessons learnt regarding SUMP evaluation Structured and well prepared self assessment greatly facilitates a review. The assessment needs to take national context and legislation into consideration, since cities‟ possibilities and power vary between nations. Peer reviews can be a fruitfu l instrument to assess a city‟s performance, but need thorough preparation and an organised review team. The questions and the purpose of the review need to be clearly defined in advance. A self assessment template is available through the BUSTRIP project. Before visiting a city the QUEST auditor needs to make a thorough desk review of the self assessment report. The peer review carries out a gap analysis between the actual performance of the city and the SUMP benchmark defined by the project. The peer review report follows a structured template to make comparisons possible.


A clearly structured self assessment template needs to be prepared before forming a QUEST analysis.

Reviewed by Elisabeth Undén and Michael Koucky, K&P

http://www.bustrip-project.net/


03/01/2012 125 TTR


PIMMS - Partner Initiatives for the development of Mobility Management Services,

and

PIMMS TRANSFER - Partner Initiatives for the development of Mobility Management Services and TRansferring Actions iN Sustainable mobility For European Regions

PIMMS: http://www.pimms-eu.org/

PIMMS TRANSFER: http://www.pimms-transfer-eu.org/

PIMMS: 2004-7

PIMMS TRANSFER: 2008-11

√


PIMMS:EU, INTERREG IIIC

PIMMS TRANSFER: EU, INTERREG IVC

PIMMS: London Borough of Bromley (UK); City of Graz (Austria); County Administrative Board of Stockholm (Sweden); Province of Treviso (Italy); City of Almada (Portugal); Municipality of Terrassa (Spain); Frankfurt Public Transport Authority (Germany); City of Serres (Greece)

PIMMS TRANSFER: London European Partnership for Transport (UK); City of Graz (Austria); County Administrative Board of Stockholm (Sweden); Province of Treviso (Italy); City of Almada (Portugal); Frankfurt Public Transport Authority (Germany); Erasmus University; City of Serres (Greece); Bratislava (Slovakia); Klaipėda (Lithuania); Maribor (Slovenia); Larnaca (Cyprus); Gdansk (Poland); Sofia (Bulgaria); Timişoara (Romania)

Remit PIMMS investigated how MM could be adopted more effectively in regional transport and sustainability policy. It established a methodology for benchmarking the use of MM, utilising local sustainability indicators.

PIMMS TRANSFER is an evolution of the PIMMS project. Partners include “Progression Regions”, most of the original PIMMS partners, with some experience of MM. There are also “Initiation Regions”, from new Member States, new to the concept, and targeting regional/local decision-makers to learn about good practice and its transfer back home. Project objectives are:

- to stimulate modal shift towards more sustainable forms of transport by increasing implementation of high quality MM techniques and policies in European regions

- to extend, deepen and promote best practice in mobility management.

(The project originally planned to conduct a review of SUMPs, but this task is no longer being undertaken, in order to avoid duplication with the ELTIS Plus review.)

http://www.pimms-eu.org/

http://www.pimms-transfer-eu.org/


03/01/2012 126 TTR

PIMMS/PIMMS TRANSFER


Both projects are of limited relevance, focusing on transfer of good practices relating to MM, rather than SUMP evaluation specifically. However, partners undertook two activities which are of interest:

- identifying Critical Success Factors (CSFs) to help measure success of different aspects of MM and inform future policy development/measure implementation/transfer of good practice

- a “self-benchmarking” exercise, whereby partners compared their performance in different aspects of MM with other organisations.

Lessons learnt regarding SUMP evaluation No significant lessons reported, given coverage of the two projects. The importance of identifying CSFs is emphasised, which can be used as a benchmarking tool.


We need to consider how benchmarking will be addressed within the QUEST audit process, and also whether the audit process should identify CSFs.



03/01/2012 127 TTR


OPTIMUM II - Optimal Planning Through Implementation of Mobility Management

http://www.optimum2.org 2008 -10 √√

Funding programme Partners and other stakeholders involved/Authors

INTEREG III – North West Europe Lead Partner: Provincie Noord-Holland

Other Partners:

UK - Essex County Council

UK - Lancashire County Council

K - London Borough Council of Southwark

UK - City of Edinburgh Council

UK - Royal Devon & Exeter Healthcare NHS Trust

NL - Provincie Gelderland

NL - Provincie Zuid-Holland

NL - Novem (The Netherlands Agency for Energy and the Environment)

Remit OPTIMUM II aims to substantially increase the effects of MM by focusing upon the five key factors (the OPTIMUM II pillars) that until now have been insufficiently addressed in MM programmes:

- user-oriented approach - taking on MM from the start - communication and information - marketing and promotion - enforcing MM measures.

Approach to SUMP evaluation Mainly the project intends to formulate some guidelines to implement MM in a better way in the planning process of urban areas, especially in relation to land-use planning. It is not clear whether they will develop some specific analysis approach to assess the results of this planning process.

Lessons learnt regarding SUMP evaluation Need to pay also attention to MM issues.

http://www.optimum2.org/


03/01/2012 128 TTR

OPTIMUM II


N/A

Reviewed by Dirk Engels, Tritel


03/01/2012 129 TTR


CO2NeuTrAlp - CO2-Neutral Transport for the Alpine space

http://www.co2neutralp.eu/ 2008-11 √√


European Territorial Cooperation / European Regional Development Fund

BAUM Consult; AllgäuNetz; Comune di Torino; Comune di Padova; Dolomiti Bus; FGM-AMOR; Holding Graz; German Research Centre for Environmental Health; Parco Nazionale Cinque Terre; Provincia di Belluno; Provincia di Brescia; Litija Development Centre; Rhônalpénergie (Environnement); Università Bocconi CERTeT; University of Maribor.

Remit This project explores the feasibility of the use of locally-sourced renewable energy being used for both private and public transport, as an alternative to fossil fuels. The project has a particular geographical focus on the Alpine Region, which comprises five countries. The work is based on the demonstration and testing of 13 pilot projects. These pilot projects encompass sustainable transport in both rural and urban contexts, and also include both passenger and freight transport.

The key guidelines from the project are contained in “Renewable Energies in Transport: Local Action to Promote the Shift towards the Age of Solar Mobility”, which is available as two separate documents – “Guidelines for Decision Makers”, which is aimed at decision makers in local and regional authorities, and “Guidelines for Transport Professionals”. These documents provide practical advice on how to plan and implement sustainable transport and mobility services.

Approach to SUMP evaluation Whilst there is no specific reference to SUMPs in either document, there is practical advice on how to involve stakeholders in the planning process. A “participative planning tool” is provided, with precise guidance on how to involve important stakeholders in the implementation of new, sustainable services. This includes a description of an eight-step procedure for establishing a co-operative Local Implementation Network. Evaluation of the pilot projects featured is undertaken using the MaxEva Evaluation Tool, (which is being examined by other QUEST partners).

There is a detailed discussion of criteria that should be used to inform decision making with regard to the use of renewable energy sources, and this is effectively a discussion of indicators that should be used for monitoring and evaluation. Among the criteria recommended are the reduction of greenhouse gas emissions, the net reduction of toxic emissions and the elimination of unnecessary trips. Criteria recommended are grouped under the headings: Environmental, Technical, Economic, Social and Spatial Development.

Lessons learnt regarding SUMP evaluation As the project used the MaxEva Evaluation Tool, there is probably little to add, from here, to the review of MaxEva itself, which is being carried out elsewhere.

http://www.co2neutralp.eu/


03/01/2012 130 TTR

CO2NeuTrAlp


See above.



03/01/2012 131 TTR

Sustainable Urban Transport Project


SUTP - Sustainable Urban Transport Project

Weblink:

http://www.sutp.org (http://www.sutp.cn/ for China)

GIZ SUTP has developed a Sourcebook on Sustainable Urban Transport for Policymakers in Developing Cities, and training documents that complement this effort (most of them translated to Chinese and Spanish, and some to other languages). After registration (at no charge), users can download all these and other documents from the website.

Training documents:

http://www.sutp.org/index.php?option=com_docman&task=cat_view&gid=97&Itemid=54&Itemid=196

2003-present √


GIZ SUTP Asia

GIZ SUTP Latin America

GIZ SUTP Africa

Key organisations involved (other than GIZ- Deutsche Gesellschaft für Internationale Zusammenarbeit):

Asia: UNESCAP, CITYNET, BMA, UN-HABITAT.

Latin America: ARPEL , CAF , TransMilenio and UN-HABITAT .

Africa: Sub-Sahara African Transport Policy Program (SSATP)

Office Locations: New Delhi, Bangkok (for Asia), Bogotá (for Latin America), none in Africa

http://www.sutp.org/

http://www.sutp.cn/



http://www.unescap.org/

http://www.citynet-ap.org/

http://www.bma.go.th/

http://www.unhabitat.org/

http://www.arpel.org/

http://www.caf.com/

http://www.transmilenio.gov.co/

http://www.unhabitat.org/

http://www.worldbank.org/afr/ssatp/


03/01/2012 132 TTR

SUTP

Remit

SUTP‟s main objective is to assist developing world cities achieve their sustainable transport goals, through the dissemination of information about international experience, policy advice, training and capacity building and targeted work on sustainable transport projects within cities.

The Project has three main branches in Asia and Latin America. In Asia (SUTP- Asia), the project is carried out in cooperation between Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ), the United Nations Economic and Social Commission for Asia and the Pacific (UNESCAP), CITYNET, UNHABITAT and the Bangkok Metropolitan Administration (BMA). Offices are located in New Delhi (India) and Bangkok (Thailand). In Latin America, the project receives cooperation from Cooperación Andina de Fomento (CAF), ARPEL, UNHABITAT and TransMilenio and is based in Bogotá (Colombia). There are also selected activities in Africa, though the project does not have an office in that continent yet.

Approach to SUMP evaluation Work has included the technical assessment of urban transport policies and projects in specific cities.

The report Sustainable Urban Transport Technical Document 7: Sustainable Transport Evaluation reviewed evaluation schemes in the context of the high-level UN CSD process. However, this has a focus on pre-SUMP activities such as indicators and data collection rather than the downstream assessment of the implementation and monitoring of SUMP-like strategies.

Lessons learnt regarding SUMP evaluation N/A


Annex II of the Sustainable Urban Transport Technical Document 7: Sustainable Transport Evaluation describes a number of indicator-based projects, including:

- ADB Indicators to Measure Sustainable Transport (excluded public participation): aimed to help decision makers to develop more structured and quantified approaches to policymaking

- SloCat Initiative: comprehensive list of sustainability indicators - Urban Transport Benchmarking Initiative: compared the transport systems of 44 participating cities (see separate

inventory entry - OECD Indicators for the Integration of Environmental Concerns into Transport Policies - UITP Evaluation of Sustainability in Public Transportation: a self-assessment tool for public transport operators.

http://www.sutp.org/dn.php?file=TD-STE-EN.pdf

Reviewed by Jürgen Perschon and Simon Field, EURIST

http://www.sutp.org/dn.php?file=TD-STE-EN.pdf


03/01/2012 133 TTR


SUTIP Indonesia -

Sustainable Urban Transport Improvement Programme

http://www.sutip.org

2009-present √


The project is co-financed by the German Federal Ministry for Environment, Nature Conservation and Nuclear Safety (BMU) in the context of the International Climate Protection Initiative.

GIZ, Indonesian Ministry of Transport, 4 partner cities: Palembang, Bogor, Solo and Yogyakarta.

Remit Helping Indonesian cities to implement environmentally friendly and energy efficient transportation schemes. Project Updates (available on project website: http://www.sutip.org):

- Regulator Coaching WS in Bogor- Indonesia - Transport Ministry of Indonesia Launches 3rd Public Transport Day - the City of Palembang is moving ahead to implement sustainable urban transport schemes - the City of Solo is working on a Sustainable Urban Transport Strategy and Plans - the Indonesian Land Transport Academy - World Bank Bus Reform Toolkit (in Indonesian).


The project provides methodological and technical support in the development and implementation of measures to improve urban transport in selected medium-sized and large cities with over 100,000 inhabitants. In order to address the constraints in urban transport, the stakeholders propose a two-tier approach to assist the central and local government agencies at the same time. Taking into account the current institutional and jurisdictional setup in Indonesia, the strategic approach for the project covers the two components:

1. Support for the national co-ordinating body

2. Support to selected city governments

http://www.sutip.org/

http://www.sutip.org/

http://www.sutp.org/index.php?option=com_content&task=view&id=2729

http://www.sutp.org/index.php?option=com_content&task=view&id=2694

http://www.sutp.org/index.php?option=com_content&task=view&id=2591&Itemid=1&lang=en




03/01/2012 134 TTR

SUTIP Indonesia

Approach to SUMP evaluation Throughout the project, capacity development with the objective to strengthen the capacities of involved decision makers and transport planners is provided. The project also assists the Ministry of Transportation (MoT) in the formulation of instruments for integrated transportation. This strengthens the MoT in its role in guiding, supporting and supervising the cities to develop and implement their own action plans.

The second component, dedicating the overwhelming part of the project resources towards work with the selected cities, is to provide direct assistance for city governments. The component is aimed at supporting the city administrations in implementing certain improvement measures that lead to environmental friendly and more energy efficient transport means, also co-benefiting the move towards meeting the climate goals. This will especially benefit the selected cities and pave the way for more liveable surroundings.

Direct support for cities in implementing Sustainable Urban Transportation concepts includes "Municipal Transportation Development Action Plans". Action Plans identify priority areas in each city, align existing transportation projects within cities and neighbouring districts, and they align cities' transport projects with national policy for better coherence. They have measurable impacts for long, medium and short-term measures. The work plans were drawn up with the cities during March 2010.

Lessons learnt regarding SUMP evaluation The implementation phase of SUTIP which is part of the Environment and Climate Change programme, started mid 2009. Various documents have been prepared, trainings executed, advice provided on transport legislation and related regulations, and inputs prepared for the National Road Map on CO2 mitigation, for the MoT. Work so far has been done with 4 partner cities- namely Palembang, Bogor, Solo and Yogyakarta and various trainings, advice on public transport restructuring, TDM measures and NMT development is being provided on an ongoing basis.

The impact status of May 3rd

2011 is explained here: http://www.sutip.org/project-progress.


None.

Reviewed by Jürgen Perschon, EURIST

http://www.sutip.org/project-progress


03/01/2012 135 TTR


SUMA - Sustainable Urban Mobility in Asia http://www.cleanairnet.org/suma Ongoing √


SUMA is co-funded by the Swedish International Development Cooperation Agency

SIDA

- Clean Air Initiative for Asian Cities (http://www.cleanairnet.org/caiasia) - Asian Development Bank (http://www.adb.org) - EMBARQ-the World Resources Institute Center for Sustainable Transport (http://embarq.wri.org) - GIZ Sustainable Urban Transport Project (http://www.sutp.org) - Interface for Cycling Expertise (http://www.cycling.nl) - Institute for Transportation and Development Policy (http://www.itdp.org) - United Nations Center for Regional Development (http://www.uncrd.or.jp/env/est).

Remit 7.1 To help Indonesian cities implement environmentally friendly and energy-efficient transport schemes, thereby contributing to sustainable urban development.

7.2 Example activities:

- training course on Bus Rapid Transit in Pimpri Chinchwad in November, 2009

- mass Transit and BRT planning two-day training course

- 2-day and 6-day training courses on Mass Transit in Indore (India)

- 2-day and 6-day training courses Mass Transit training held by GIZ SUTP under SUMA-Project

- Specialised Training course on NMT Planning held in New Delhi under SUMA

- TDM training course held in Singapore.

GIZ-SUTP conducted “Train-the-Trainer” initial course in Beijing, China.

A number of pre-feasibility studies for specific transport projects and packages of measures have been produced.


SUMA works with Asian countries and cities to strengthen the formulation and implementation of sustainable urban transportation policies.

The project has provided support for the development and implementation of measures to improve urban transport in medium-sized and large cities. Interestingly this has targeted both the national government department for transport as well as local governments. Ultimately the project has/will help participating cities to develop “Municipal Transportation Development Action Plans”. These Action Plans identify priority areas in each city, align existing

http://www.cleanairnet.org/suma

http://www.cleanairnet.org/caiasia

http://www.adb.org/

http://embarq.wri.org/

http://www.sutp.org/index.php?option=com_content&task=blogcategory&id=104

http://www.cycling.nl/

http://www.itdp.org/

http://www.uncrd.or.jp/env/est/

http://www.sutp.org/index.php?option=com_content&task=view&id=1885&lang=en








03/01/2012 136 TTR

SUMA


transportation projects within cities and neighbouring districts, and they align cities‟ transport projects with national pol icy for better coherence. They have measurable impacts for long, medium and short-term measures.

Taking the example of Palembang, the overall approach to reforming urban transport policymaking was to “develop [the existing] broad [infrastructure-based predict & provide] policies into a more detailed and co-ordinated urban transport strategy, which includes [a long-term strategy], more detailed policy statements, a medium-term investment programme and a medium-term action plan”. A QM approach was recommended, focusing on the conversion of vague policy statements into practical policies designed to address identified problems and contribute to achieving the overall vision. This and the need for policy co-ordination across sectors, institutions and timeframes were brought together under an Urban Transport Policy Framework.

See http://sutip.mine.nu/Palembang/CDIA/Final_report/CDIA_Palembang_Final_Report.pdf (March 2011).

As with many such strategies, it has not yet been implemented, monitored and evaluated.

Lessons learnt regarding SUMP evaluation Data not available.


None.

Reviewed by Jürgen Perschon and Simon Field, EURIST

http://sutip.mine.nu/Palembang/CDIA/Final_report/CDIA_Palembang_Final_Report.pdf


03/01/2012 137 TTR


SUTECA - Sustainable Urban Transportation in Eastern Europe and Central Asia (Focus Ukraine)

http://www.mobilnist.org.ua/ (redirects from http://www.sutp.org/suteca ).

2006-present √


The project is co-financed by the German Federal Ministry for Environment, Nature Conservation and Nuclear Safety (BMU) in the context of the International Climate Protection Initiative.

Lviv, CIFAR, UNITAR, GIZ.

Note: website is poorly maintained, with no overall project goals listed and newsletters available only in Russian/Ukrainian.

Remit

7.1 SUTECA, which started as a forum for discussions on sustainable transport in eastern Europe and central Asia, has evolved into the country-specific Sustainable Mobility in Ukrainian Cities project.

The overall vision and mission is to “develop intelligent concepts by using international experiences, and to implement these measures in order to create new modern transport systems and a better mobility for residents and visitors of Ukrainian cities”. Specific areas of work include:

- mobility management and transport planning

- event reports

- public passenger transport

- cycling and walking.

Unfortunately detailed information is only available in the local language: see

http://www.mobilnist.org.ua/en/downloads.html.

http://www.mobilnist.org.ua/

http://www.sutp.org/suteca

http://www.mobilnist.org.ua/en/downloads/cat_view/54-mobility-management-and-transport-planning.html

http://www.mobilnist.org.ua/en/downloads/cat_view/50-event-reports.html

http://www.mobilnist.org.ua/en/downloads/cat_view/43-public-passenger-transport.html

http://www.mobilnist.org.ua/en/downloads/cat_view/38-cycling-and-walking.html

http://www.mobilnist.org.ua/en/downloads.html


03/01/2012 138 TTR

SUTECA


Through its CIFAL training centres UNITAR offers workshops on challenges and approaches in SUT targeted at Mayors, local authorities, directors of technical services and representatives of other governmental and non-governmental entities. GTZ SUTP supports these workshops on a regular basis through presentations and the dissemination. Key issues of the SUTECA work are:

- Encourage knowledge-sharing between municipal and regional practitioners in the field of sustainable urban transport

- Disseminate expertise in the fields of:

o financing public transport o fleet modernisation o integrated transport management o private sector participation o awareness raising and multi-stakeholder consultations o sustainable modes of transport.

Lessons learnt regarding SUMP evaluation Since 2006 several successful workshops on sustainable urban transport were held:

- Workshop: International Exchange of Experiences on Development of Cycling Infrastructure

- Lviv Trolley Marketing Symposium

- Working Together for Sustainable and Healthy Urban Transport" workshop in Kyiv

- VELOFORUM Ukraine with GIZ participation

- Podium Discussion: “Kyiv Transport Infrastructure: City for the people or for the cars?”

No further information available.


None.

Reviewed by Jürgen Perschon, EURIST

http://www.mobilnist.org.ua/en/news/project-news/113-giz-participated-in-qworking-together-for-sustainable-and-healthy-urban-transportq-workshop-in-kyiv.html

http://www.mobilnist.org.ua/en/news/project-news/110-veloforum-ukraine-with-giz-participation.html

http://www.mobilnist.org.ua/en/news/project-news/103-podium-discussion-kyiv-transport-infrastructure-city-for-the-people-or-for-the-cars.html


03/01/2012 139 TTR

National SUMP guidelines


Mobility Handbook with additional guidelines on the website of the Flemish Government explaining the Flemish Mobility Decree of 20.4.01 amended on 20.3.09 on Mobility

http://www.mobielvlaanderen.be 2001 - 2010 √√

Funding programme Author / Other stakeholders involved

N/A Author: Flemish Administration

Stakeholders: Department of Traffic and Road, the Flemish Transportation Company (De Lijn) , Provinces

Remit Guidelines on the process and content of a SUMP for Flemish Communes and conditions for funding the studies and implementation of measures. Plans are made by the commune in consultation with all stakeholders.

Approach to SUMP evaluation Process evaluation (making the plan): documents are presented for the Provincial Audit Committee (PAC). This control organism consists of an external independent auditor and the stakeholders. In the first phases the PAC checks if the right actions to analyse the mobility problems and to define sustainable scenarios are planned. In the last phase the PAC controls whether the “Policy Plan” :

- is structured in the required way e.g. in an information part and a directive part - consists of all necessary elements: proper objectives, measures on land-use development and a hierarchic road

structure, measures for all mode networks and accompanying measures, evaluation and monitoring, an action programme

- was subject of a participatory process or a public hearing.

Impact analysis: In the Mobility Plan an overview of monitoring actions and evaluation analyses is described. Also a timing of meetings of the Accompany Committee (the commune with the Stakeholders) and of updates of the Action Plan is given.

http://www.mobielvlaanderen.be/


03/01/2012 140 TTR

Flemish Mobility Handbook with additional guidelines


The question remains how we evaluate the quality of the measures of the plan: do they react on the right problems, do they work on the right themes?

The evaluation of the impact of the plan is not clearly developed yet: a need for clear objectives for the planned actions, (quantifiable targets) and indicators to evaluate the progress.


N/A

Reviewed by Natalie Craeghs, Tritel


03/01/2012 141 TTR


Consultation in Urban Mobility Plans: Why? With whom? How?, CERTU, Lyon, 2003

La concertation dans les PDU: pourquoi? Avec qui? Comment?, CERTU – Transport et mobilité, dossier n° 146, Lyon.

2003 √√


N/A CERTU – Département Mobilité et Déplacements

Remit Guidelines for setting up a public consultation for SUMPs, adapted to the local context. The document contains information sheets on the different utensils that can be used for providing information, building a dialogue and tracing the public opinion.

Approach to SUMP evaluation This document sums up the most important aspects of a public consultation, containing the relevant elements to choose the most appropriate form of public consultation (stakeholders, representativeness, public consultation or institutional consultation, consultation on different levels of public authority).

Lessons learnt regarding SUMP evaluation This document focuses on the evaluation of the process of making a SUMP, more specifically on the requirements on public consultation. The LOTI (Law on the Orientation of Internal Transports) specifies a number of requirements for the elaboration process of the SUMP:

- association of a number of actors - consultation of public instances - consultation of the affected population.

To evaluate the impact on the environment, an environmental report must be elaborated and is subject to the consultations above as well. However, the advice of the consulted public instances is not binding.


The document provides clear guidelines on the process of consulting public instances and citizens. However, the evaluation of the Impact of the plan is not clearly developed yet: a need for clear objectives for the planned actions, (quantifiable targets) and indicators to evaluate the progress. Furthermore, it is not obligatory to adjust the SUMP to the given advice on the environmental impact of the plan.

Reviewed by Eliene Van Aken, Tritel


03/01/2012 142 TTR


Guidance on LTPs http://www2.dft.gov.uk/adobepdf/165237/ltp-guidance.pdf 2009 √√


UK government, Department for Transport (DfT)

DfT

Remit This guidance document outlines the policy framework for LTPs and describes key steps in developing and delivering quality LTPs.

Approach to SUMP evaluation Of the 48 page document, reference to evaluation of LTPs starts on page 33. There is no clear process or method of evaluation of LTPs outlined in this document though reference is given to indicators identified in the DISTILLATE project. “Guidance on the development of a monitoring strategy and the selection of indicators is available from the DISTILLATE project and included within the Local Transport Planning Network”

Paragraphs 59-60: Local authorities should consider as they develop their Plan what performance indicators are most appropriate for monitoring it, and what targets might be set to incentivise and secure delivery. Performance monitoring should be an integral part of managing the LTP programme. A strong LTP will include ambitious target setting, clear trajectories and close monitoring of delivery. Local authorities are encouraged to discuss with other authorities, especially within their region, what standard indicator definitions may enable them and the wider transport community to benchmark their performance.

Lessons learnt regarding SUMP evaluation Evaluation needs to be planned carefully before SUMPs are taken forward, so that processes can be put in place to collect the necessary data. Results should be shared across organisations responsible for SUMPs, to help build a knowledge base. See also DISTILLATE project review.


Evaluation arrangements should be considered early within the process of SUMP development. Emphasis on use of indicators/benchmarking and exchange of results with peers. See also DISTILLATE project review.


http://www2.dft.gov.uk/adobepdf/165237/ltp-guidance.pdf


03/01/2012 143 TTR

N.B: A table on p.22 of the ELTIS PLUS state of the art report on SUMPs indicates the existence of national SUMP guidance produced by various EU states. All these references were followed up, however:

- German SUMP guidance has not been published yet. A working group at the Forschungsgesellschaft für Straßen- und Verkehrswesen (FGSV) is currently drafting suggested guidance, to be published in 2012

- it has not been possible to locate any Norwegian SUMP guidance.


03/01/2012 144 TTR

Other European/national projects, initiatives and documents


Urban Transport Benchmarking Initiative http://www.transportbenchmarks.eu/ 2003-6 √


EU Project leader : Transport & Travel Research Ltd; International Association of Public Transport (UITP); The Regional Environmental Centre for Central and Eastern Europe (REC)

Remit The project benchmarked different aspects of 45 participating European cities‟ transport systems, with themed working groups each researching individual urban transport topics in detail. The project focus was to develop practical and workable indicators in order to enable strengths and weaknesses of transport systems to be assessed. Project participants of working groups identified 6 themes to be researched:

- Behavioural and Social Issues in Public Transport - City Logistics - Cycling - Demand Management - Public Transport Organisation and Policy - Urban Transport for Disabled People.

Benchmarking is a practical tool which involves: comparing performance; understanding differences and identifying best practice. The Benchmarking concept provides local and regional “actors” with a means of sharing knowledge and experiences in order to provide each of them with the knowledge to improve their transport system

In order to compare urban transport systems a range of indicators were devised. Site visits organised to assist in the exchange of best practice examples.

Approach to SUMP evaluation Benchmarking against other cities according to process outlined above.

Lessons learnt regarding SUMP evaluation Comparisons with other cities including site visits too.

Recommendations about urban transport best practices were made by a network of urban transport operators, user groups, local authorities and municipalities, rather than a single centralised institution.

http://www.transportbenchmarks.eu/


03/01/2012 145 TTR

Urban Transport Benchmarking Initiative


Value of site visits and information exchange.

Because of the number and range of participants the process of Urban Transport Benchmarking was very fluid, responding to the issues which were raised by urban transport stakeholders, rather than following a rigid process which was predetermined.



03/01/2012 146 TTR


SMOOTH http://www.smooth-project.eu/ See below √


EU Project co-ordinated by, Greater Community of Cahors (France), in partnership with: Ministry of Communications and Works, Nicosia (Cyprus); Municipality of Cahors (France); Vallons de Vilaine County (France); Municipality of Kaunas (Lithuania); Quercy Energies (France); Regional Agency for Environment in Midi-Pyrenees (France); Euro Project Consult (France).

Remit The overall objective of SMOOTH is to reduce car use in four territories representative of the enlarged Europe by enabling a real choice of multimodal transport options, including safe walking, cycling and on-demand transport.

The project will also cover initiatives to promote car-pooling.

It is reported that SMOOTH is inspired by the organisation and the conclusions from the MOVE IEE-STEER project (http://www.move-project.org/). (MOVE is an EU-project about organising and improving our future

Mobility.)

WP6 will gather all evaluation activities with the objective of providing the framework for a common assessment of the action.

Approach to SUMP evaluation Due to lack of detail on the project website, two requests were made for information about WP6 outputs and the project timescale of project. No replies were received, so it is not possible to specify the approach to SUMP evaluation. The project is graded as being of limited relevance to QUEST based on currently available information.

Lessons learnt regarding SUMP evaluation Not possible to specify (see above).


Not possible to specify (see above).


http://www.smooth-project.eu/

http://www.move-project.org/


03/01/2012 147 TTR


COST 356. EST – Towards the Definition of a Measurable Environmentally Sustainable Transport.

Final Report: Joumard R, and Gudmundsson H, “Indicators of Environmental Sustainability in Transport: an Interdisciplinary Approach to Methods” (May, 2010).

http://cost356.inrets.fr/ and http://hal.archives-ouvertes.fr/hal-00492823/fr/

2005 -10 √


European Co-operation in Science and Technology (COST). This is an intergovernmental framework, supported by the EU RTD Framework Programme, designed to co-ordinate nationally funded research at a European level.

There are 36 European countries that make up COST. It would appear that the French research institution, INRETS (Institut National de Recherche sur les Transports et leur Sécurité), had a prominent role in COST 356. The Chairman of COST 356 was Dr Robert Joumard, of INRETS.

Remit The objective of COST 356 was to bring together and review current European knowledge on the assessment of the environmental sustainability of transport. A major aim was the identification and harmonisation of scientifically sound assessment methods, in order to facilitate the building of a set of indicators that can be used for the evaluation of transport projects, plans and policies.

Approach to SUMP evaluation Although there is no specific reference made to SUMPs, this COST Action addresses the fundamental issues of how the environmental impacts of transport can be measured, and which indicators should be used in planning and decision making.

Lessons learnt regarding SUMP evaluation The work includes a discussion of the precise definition of “environmental sustainability in transport”, and develops the concept of the “chain of causality” in relation to impacts on the environment, with seven examples of such “chains” defined. COST 356 has also seen the development of criteria and methods for the assessment of indicators of sustainable transport, in terms of measurement, monitoring and management. The indicators discussed are divided into three categories: Environmental Indicators, Sustainability Indicators and Sustainable Transport Indicators. Five specific case studies are used, to illustrate the use of indicators in the assessment of environmental impacts of transport projects.

http://cost356.inrets.fr/

http://hal.archives-ouvertes.fr/hal-00492823/fr/

http://hal.archives-ouvertes.fr/hal-00492823/fr/


03/01/2012 148 TTR

COST 356


This is a very detailed document, consisting of 426 pages. It goes into detailed discussions of the nature of decision making, the definition of sustainability in the context of transport and the selection of indicators.

An interesting suggestion is that, nowadays, with an increasing emphasis on stakeholder engagement in the planning process, accurate data relating to quantitative indicators are becoming less important as a basis for informing rational decision-making. We should consider how true this is in relation to the evaluation of SUMPs.

There are two other COST Actions that relate to transport and the environment: COST 350 “Integrated Assessment of Environmental Impact of Traffic and Transport Infrastructure”, which looks at integrating environmental aspects of traffic and land transport infrastructure at a regional level, and COST 355 “Changing Behaviour Towards a More Sustainable Transport System”, which analyses how the behaviour of travellers, shippers and carriers can be changed, in order for the growth in the demand for unsustainable means of transport to be reversed.

Reviewed by Phil Barham,TTR


03/01/2012 149 TTR


Energy Cities (Covenant of Mayors) http://www.energy-cities.eu/ 2009-11 √


From 2009 to 2011, Energy Cities is under the Presidency of the City of Heidelberg (Germany) with a Board of Directors of 11 European cities. The association created in 1990 represents now more than 1,000 towns and cities in 30 countries.

Remit Energy Cities is the European Association of local authorities inventing their energy future. The main objectives are to strengthen roles and skills in the field of sustainable energy and to represent interests and influence the policies and proposals made by EU institutions in the fields of energy, environmental protection and urban policy. Additionally, to develop and promote initiatives through exchange of experiences, the transfer of know-how and the implementation of joint projects. There are a number of projects; many are dissemination sharing opportunities to promote sustainable energy.

Of possible interest to QUEST is the ENGAGE project which aims to directly support the EU‟s Energy & Climate Package targets by involving local authorities who signed the Covenant of Mayors (not exclusively).The objective is to provide an efficient participative PR campaign strategy and an innovative, ready to use on-line tool in order to render the participation of the civil society not only feasible but also desirable. ENGAGE aims to mobilise at least 3,300 stakeholders and citizens and display each tangible engagement via publicly visible posters created thank to the on-line tool. Part of the engagements can also be evaluated with a CO2 and energy savings calculation tool.

Approach to SUMP evaluation Exchange of experiences, the transfer of know-how and the implementation of joint projects.

Lessons learnt regarding SUMP evaluation Nothing specific regarding SUMP evaluation, though the networking and exchange of experience as a result is a valuable attribute of this initiative, particularly the online tool developed in ENGAGE.


Emphasis on network of association and shared dissemination of results.


http://www.energy-cities.eu/

http://www.energy-cities.eu/-Board-of-Directors,455-

http://www.energy-cities.eu/-Board-of-Directors,455-

http://www.energy-cities.eu/-Members-


03/01/2012 150 TTR


DISTILLATE - Design and Implementation Support Tools for Integrated Local land Use, Transport and the Environment

http://www.distillate.ac.uk/about/about.php

2004-2008

(taken from dates of output reports-inception & final)

√√


UK Engineering and Physical Sciences Research Council‟s overarching research programme on the development of a Sustainable Urban Environment

DISTILLATE aimed at overcoming the barriers to the effective development and delivery of sustainable urban transport and land use strategies.

Partners: Transport Research Laboratory; Institute for Transport Studies (University of Leeds); SEI; University College London; Heriot

Watt University

Local Authority Partners: Bath and North East Somerset Council; Blackpool Borough Council;

Bristol City Council; Essex County Council; Leeds City Council ;MerseyTravel PTE ;Nottingham City Council ;

Sheffield City Council; South Yorkshire PTE; Stockport Borough Council; Strathclyde PTE ;Surrey County Council West Yorkshire PTE (Metro) ;City of York Council ;Yorkshire and Humber Regional Assembly

Remit

The principal objective of DISTILLATE was to develop, through a focused, inter-disciplinary research programme, ways of overcoming the barriers to effective development and delivery of sustainable urban transport and land use strategies and, through them, enhanced quality of life.

DISTILLATE was divided up into seven projects which covered a range of different activities involved in the development of sustainable urban transport and land use strategies:

Project A - Organisational Behaviour and Barriers

Project B - Improved Tools for Option Generation

Project C - Improved Indicators for Sustainable Transport & Planning

Project D - Improved Effectiveness in Organisational Delivery

Project E - Improved Mechanisms for Funding and Phasing of Implementation

Project F - Enhanced Analytical Decision Support Tools

Project G -Enhanced Appraisal Tools

http://www.distillate.ac.uk/about/about.php


03/01/2012 151 TTR

DISTILLATE

Remit

Project C: The research brief for the indicators specification was to:

- conduct a survey of local authorities‟ experience in measuring, predicting and using indicators; - determine the extent to which current indicators correspond to stakeholders‟ understanding of sustainability and

quality of life; - specify the requirements for a core set of indicators at each stage in the decision-making process; and - identify a core set of outcome indicators that best meets those requirements. Deliverable C1 Sustainable

Transport Indicators: Selection and Use http://www.eltis.org/docs/DISTILLATE_Sustainable.pdf

Project G: The final product took the form of an assessment matrix which allows users to select a set of indicators appropriate to their circumstances, weight the indicators in terms of importance, assess the impact of proposed project(s) against the indicators, and finally combine the weighting and assessment to derive a final score that will give an indication of whether a project is worth pursuing, or a priority list where more than one project is assessed. Packages of measures could also be assessed. A pick list of potential indicators will be provided derived from NATA, Local Transport Plan (LTP) guidance, Best Value indicators and other appropriate sources. Users will also be able to add their own local indicators - these may reflect specific local issues in the LTP or even regional strategies. Indicators are likely to be weighted in line with the LTP, but different priorities can be set if appropriate. Guidance on selecting indicators, weighting and assessing the impact of projects will be provided to accompany the matrix. Likely users include local authority decision makers, although the matrix could also be used as part of the consultation processes with stakeholders. In the first instance the matrix is designed to support the decision making process for small schemes, but it is also intended that it could be used with larger schemes to provide an initial indication of whether they are worth pursuing to the full appraisal stage.

There are additional tools for representing and appraising the distributional impacts of policies. For more information on enhanced appraisal tools: http://www.distillate.ac.uk/outputs/Deliverable%20G1%20ver%203.0.pdf

Approach to SUMP evaluation Whilst this is not a SUMP evaluation per se, it does demonstrate an approach that uses a number of different tools to assist in evaluating and assisting a decision making processes.

Lessons learnt regarding SUMP evaluation The motivation for project G in DISTILLATE came from local authority concerns about appraisal and their perception of appraisal as a barrier to the development of sustainable transport and land use policies. In relation to QUEST this is the value of DISTILLATE outputs i.e. tools used to assist in decision making.


Use of a range of analytical tools, including indicators, specific to local need and local situation to assist in decision making processes.


http://www.eltis.org/docs/DISTILLATE_Sustainable.pdf

http://www.distillate.ac.uk/outputs/Deliverable%20G1%20ver%203.0.pdf


03/01/2012 152 TTR

N.B.:

- there is a Handbüch für Kommunale Verkehrsplanung (Handbook of Local Transport Planning). This was not reviewed as it was not freely available, and the cost of purchase was prohibitive

- a number of health-related sources were found, published by the World Health Organisation, National Institute for Health and Clinical Excellence (UK) These were not considered further due to lack of relevance

- various other national documents were identified, but not investigated further due to the additional time that would be needed for translation:

o Sweden, Trafik för en Attraktiv Stad Utgåva 2 (Traffic for an attractive city), http://www20.vv.se/vag_traf/vgu-trast/trast/index.htm o Spain, http://www.idae.es/index.php/mod.documentos/mem.descarga?file=/documentos_10251_Guia_PMUS_06_2735e0c1.pdf o Portugal, http://www.conferenciamobilidade.imtt.pt/pacmob/guia_pmts/Guia_para_a_elaboracao_de_PMT_Marco_2011.pdf o Poland, http://www.igkm.pl/site/wiadomosci,2881,polecamy.html.

http://www20.vv.se/vag_traf/vgu-trast/trast/index.htm

http://www.idae.es/index.php/mod.documentos/mem.descarga?file=/documentos_10251_Guia_PMUS_06_2735e0c1.pdf

http://www.conferenciamobilidade.imtt.pt/pacmob/guia_pmts/Guia_para_a_elaboracao_de_PMT_Marco_2011.pdf

http://www.igkm.pl/site/wiadomosci,2881,polecamy.html


03/01/2012 153 TTR

Appendix 4

Suggested questionnaire to guide interviews with individuals involved in SUMP evaluation

Interviewer‟s introduction

QUEST (Quality management tool for Urban Efficient Sustainable Transport) is a new EU project, funded by IEE. Over 30 months, QUEST will develop an audit tool to evaluate and improve sustainable urban mobility plans (SUMPs) across 45 European cities and towns. A key project task is to review current and previous approaches to SUMP evaluation. QUEST is currently investigating relevant projects, reports and guidelines from across Europe. However we do not want to rely solely on periodic project outputs, and are also interviewing individuals from organisations across Europe that implement and evaluate SUMPs.

Your interview should last no more than 15-20 minutes. QUEST will produce a report on the state of the art in sustainable urban mobility assessment. This report will be sent to the EC and will be publicly available. The report will specify the types of organisation and EU states in which interviewees are based. However the names of the interviewees will not be included. Therefore we will report anonymously any interviewee comments or common issues identified.

Questions

Name:_________________________________________________

Employing organsiation:_________________________________________________________

Type of organisation (e.g. municipality, transport authority, consultancy):

_________________________________________________________

Role in organisation:_______________________________________

Briefly describe the status of your organisation‟s SUMP. For example, how long has your organisation had a SUMP and how often is it renewed ?


03/01/2012 154 TTR

Who undertakes the SUMP evaluation? Are external stakeholders involved, if so who and how?

What level of resources (budget/staff time) are allocated for SUMP evaluation?

Briefly describe your organisation‟s approach to SUMP evaluation. For example, frequency of evaluation and method(s) used.


03/01/2012 155 TTR

What has worked well/not so well during the evaluation process?

What key lessons have been learnt regarding SUMP evaluation?

Any recommendations for other cities/towns undertaking SUMP evaluation?


03/01/2012 156 TTR

APPENDIX 5 –

Detailed inventory of projects using TQM to develop audit tools

Key

Sources are categorised by relevance.

√ = limited relevance

√√ = of some relevance

√√√ = relevant


03/01/2012 157 TTR


BYPAD - Bicycle Policy Audit http://www.BYPAD.org/

d.d. 2011-08-02

BYPAD Manual, Version 3.0: 2006 – 2008 http://BYPAD.org/docs/BYPAD-Manual.pdf

d.d. 2011-08-02

Developed by an international consortium of bicycle experts in 1999 as part of the EU project – BYPAD; two follow-up projects: BYPAD+ (2003-2005) and BYPAD-Platform (2006-2008),

√√√


EU IEE STEER Programme Vectris; Austrian Mobility Research (AMOR), IMOB – Hasselt University; Ligtermoet & Partners; Centrum Dopravniho Vyzkumu CDV; velo:consult.; ECF;

Remit

BYPAD regards cycling policy as a dynamic process where different components must fit together in order to get a well-balanced sustainable cycling policy. Because each step in this policy process has its own characteristics, BYPAD distinguishes 9 modules for which the quality of the cycling policy is determined.

http://www.bypad.org/

http://bypad.org/docs/BYPAD-Manual.pdf


03/01/2012 158 TTR

BYPAD

Remit

BYPAD “Spiral of development” for towns cities and agglomerations

The 9 fields are in permanent development and influence each other. BYPAD focuses on the actions in the field (module 5-8), on the planning and organisation of the cycling policy (module 1-4) and the monitoring of effects (module 9).

The quality of each of the nine BYPAD modules is determined separately by assigning a quality level on the four-stage ladder of development to each of them. The overall quality level of the cycling policy results from the weighted average of the quality levels of the single modules. At a glance, the ladder of development shows the strengths and weaknesses of a cycling policy.


03/01/2012 159 TTR

BYPAD

Remit

Ladder of development

For each module, quality objectives for the future cycling policy can be defined separately, and it is possible to monitor its evolution. When auditing the local cycling policy, each module is given a score based on the answers to the BYPAD-questionnaire. BYPAD distinguishes four quality levels. Each level represents a further step on the ladder of development – thus representing further progress of the local cycling policy. Quality improvement is basically climbing up the ladder of development step-wise. This means, at the same time, that a local or public authority cannot simply change to Total Quality Management overnight.


03/01/2012 160 TTR

BYPAD

Remit

Benefits for participating municipalities

- instrument enables municipalities to evaluate and improve the quality of their local cycling policy. - analyses the strengths and weaknesses in current policy - gives clear indications for future improvement. - BYPAD questionnaire is based on European best practice in cycling policy, which has been proved successful.

Measures and areas of action necessary to improve cycling policy can be derived directly from the audit results.

Dissemination

Examples of good practices are exchanged and spread through the good-practise database and/or the BYPAD-newsletter.

BYPAD regional workshops and international seminars ensure regular and intensive exchange of information between the BYPAD auditors, towns, cities and regions and all those active/interested in the field of cycling policy. Check the website regularly for the upcoming events.


03/01/2012 161 TTR

BYPAD

Description of auditing process

a. Background information enables information to be collected on the following issues:

⇒ History of cycling policy: the historical background to cycling policy in the town, city, agglomeration or region is vital information giving the auditor an idea of current and future cycling policy which will be screened in the BYPAD audit.

⇒ Cycling measures in the field: an overview of concrete cycling measures allows for a preliminary view to be obtained of current cycling policy.

⇒ Current (cycling) plans: an overview of current cycling plans or plans having an impact on bicycle use is also needed by the auditor to obtain a view of the screened cycling policy.


03/01/2012 162 TTR

BYPAD


a.1. Site visit to check out completed cycling project with cities representatives and other members of the evaluation group (not obligate)

b. Policy Audit

b.1 BYPAD questionnaire is the main instrument which need to be filled in by each member of the BYPAD evaluation group. For each includes statement based on each level of development. This is not an exhaustive list, but can be added by the participants.

b.2 BYPAD Audit Report The BYPAD audit is based on a calculation of scores given to the 9 main elements of the BYPAD method. During consensus meeting the auditor gives a score to every question, which is agreed on by the evaluation group.

c. Consensus meeting

Discussion with all participants about the different answers to the questions to seek a realistic grading of different assessments and to find the same level. The results of the consensus meeting form the basis for the evaluation group to come up with and to agree on objectives and actions for future cycling policy.

d. Cycling Quality Plan (=BYPAD Action plan)

will be set up by the auditor based on the outcome of the consensus meeting. The action plan makes the agree actions concrete by allocating a timetable, specific budgets and responsible persons and departments to the agreed objectives and actions. The BYPAD method classified the following essential elements:

1. Determining improvement goals on the basis of the audit process Setting out a priority list (you can‟t change everything at the same time)

2. Identifying the people responsible for implementing goal improvements 3. Identifying employees and departments involved. 4. Defining a time schedule for implementing objectives with opportunities for interim evaluation 5. Setting out a budget for realising the improvement goals.


03/01/2012 163 TTR

BYPAD


BYPAD evaluation group (to cover all different viewpoints):

- politicians, who are responsible for forming the local cycling policy - officials, who prepare and execute the cycling policy - user (groups), representatives of the local cyclists' organisation, municipalities.

Reach/implementation BYPAD has been carried out by over 100 cities, towns and 18 regions in 21 countries17.

More than 90 BYPAD auditors from 21 different EU countries have been trained and certified in order to guide the towns, cities and regions to implement BYPAD.

Role of the auditor BYPAD is designed as an instrument for self-auditing. Therefore municipalities screen the local cycling policy themselves.

The external (neutral) auditor supports and guides this self-auditing process. This auditor leads the session, debate and evaluates the questionnaires. The auditor creates the evaluation and improvement report.

Label or certification After the implementation of the audit the cities and regions receive from their national auditor the official BYPAD certificate together with a bicycle action plan based on the audit results and improvement steps.

The certificate recognises the efforts of the participating municipality for improving its cycling policy through total quality management. It should be seen as quality label for cycling policy. There are no further preconditions

Lessons learnt regarding development of QUEST tool

- Process oriented approach - Stakeholder acceptance and commitment necessary, but is secured by the approach.

BYPAD works best if the process is repeated ones in 2-3 years to analyse improvements. There is no evidence that municipalities do this. BYPAD doesn‟t safeguard a follow up


- Clear method but only based on one field of transport policy (cycling) – The challenge is how to widen this approach to sustainable mobility (many stakeholders, more information to be analysed, where to draw the line)

- Good format for organising TQM in transport policy - BYPAD audit scores are quite complicated. To decide if it is useful and workable for QUEST - Clear indications for the content of QUEST action plan.

Reviewed by Maja van der Voet, Ligtermoet & Partners

17 By 2008 65 cities in 15 countries had implemented BYPAD (BYPAD Manual 2008)


03/01/2012 164 TTR


Support Action for Evaluation and Monitoring of CIVITAS Plus - POINTER

http://pointer.rtd.project.net

9/6/2011

2008-13

√√


FP7, CIVITAS Plus 1 Nederlandse Organisatie voor Toegepast-natuurwetenschappelijk onderzoek TNO, the Co-ordinator

2 Gopa Cartermill S.A.,

3 University of Southampton, Transport Research Group (TRG)

4 Centrum Dopravního Výzkumu, v.v.i. (CDV)

5 Pöyry Infra Traffic GmbH

6 TRT Trasporti e Territorio Srl.,

7 Uniresearch B.V.

POINTER

Remit

CIVITAS POINTER is the Support Action to support cities and the EC with the monitoring (1) evaluation (2) of the whole CIVITAS Plus action. Objectives of CIVITAS POINTER are:

- independent formal monitoring of the project deliverables and reports from the demonstration projects and provision of specialist and independent advice to the EC

- independent overall cross-site evaluation of the measures taken by the CIVITAS Plus cities on different levels; this consists of 1) an impact evaluation, assessing achieved results in relation to the objectives and the base case (no measures) of CIVITAS Plus projects, and 2) a process evaluation, assessing achieved results and implementation processes of the CIVITAS Plus project measures in relation to the planned measures and implementation processes

- development of clear European-level policy recommendations on the basis of validated city results - dissemination of best practices on monitoring and evaluation based upon the experiences gained in CIVITAS

POINTER.

The Work plan consists of 5 work packages: WP1 Monitoring, WP2 Evaluation, WP3 Policy recommendations, WP4 Dissemination and WP0 project management. The monitoring and evaluation activities are strictly separated. The Consortium

http://pointer.rtd.project.net/


03/01/2012 165 TTR

is built on the expertise and experience of the respective partners (7 from 6 different Member States including one from Eastern Europe) on evaluating and / or monitoring of complex, international projects that are related to clean urban transport.


POINTER


The project has the WPs focused on measure evaluation (their process and impact evaluation, WP2) and policy evaluation (WP3), aiming at the formulation of guidelines and policy recommendations. Aspects to be considered have included short and long-term impacts, interactions with other policies in the transport, energy and land use planning sectors, adaptability and transferability, acceptability and costs. The results of evaluation will be translated towards the policy recommendations and part of this work will be the possibilities of transferability of measures to other cities and/or situations. One of other WPs (WP4), is focused on best practices among evaluation methods.

The measure evaluation process focuses on communication especially with the Project Evaluation Managers (PEM) which bring the compat information on measures implementation in a particular CIVITAS Plus project, then with the Local Evaluation Managers (LEM) bringing details on measures implemented in a particular CIVITAS Plus city and with the Measure Leaders (ML) summarising information on implementation of a paricular measure. POINTER uses special templates for measure evaluation which are completed by PEMs and once a while by LEMs, on status of implementation. Some measures have been pointed as “focused measures“ and they are evaluated by additional methods, such as CBA. Beside that POINTER organises regular meetings with all CIVITAS Plus PEMS, the meetings are called “ELG – Evaluation Liaison Group“ where all evaluation aspects are discussed and further guidance is provided from POINTER to PEMS / LEMs. POINTER also provides support in specific evaluation methodologies through organising thematic workshops, such as the CBA workshops, emmissions measurement / evaluation of environmental impacts of transport, BAU workshops for the business-as-usual scenarios or Learning Histories workshops etc., all these are suited to the CIVITAS Plus projects needs.

As for Policy evaluation, the action is about to start, the methodology is prepared – context data collection from the CIVITAS Plus cities (all data having connexion to transport situation), the process of getting the policy picture of CIVITAS Plus cities is continued by interviewing LEMs (particular questions on policy status, political attitudes), then the policy index per city is calculated and further analysis is performed (MCA). The aim is to benchmark the policy environment in cities and its influence on implementation of sustainable urban mobility measures, to identify all the levers activating city representatives for more sustainable policy and what are the policy solutions for facilitating such measures in practice. Finally the results of evaluation will be translated towards the policy recommendations, especially for the CIVITAS Plus cities, but also lessons for other cities.

Aspects considered in the overal evaluation will include short and long -terms impacts, interactions with other policies in the transport, energy and land use planning sectors, adaptability and transferability,acceptability etc. and part of this work will be the possibilities of transferability of measures to other cities and/or situations.

Reach/implementation The evaluation methodology is applied by all POINTER partners, but also by all the PEMs appointed by the CIVITAS Plus projects and by other evaluation specialists (LEMs). POINTER provides support for the overall evaluation of CIVITAS Plus measures – at the European scale, PEMs use the method at the project level (within the CP project consortia, across partner


03/01/2012 166 TTR

project countries), the LEMs perform it at the city level.

Role of the auditor POINTER partners play their role of evaluators, e.g. TRG is responsible for impact evaluation of CIVITAS Plus measures, TNO is in charge of process evaluation, CDV is performing policy evaluation (also communicating with the CIVITAS PAC) and POYRY is assessing the best practices of measure evaluation methods. Other partners conduct either monitoring of measures or the project management.

Label or certification CIVITAS Plus cities can be awarded by CIVITAS forum prize for their specific successes in implementation of sustainable urban mobility measures. POINTER does not provide any special certificate, just while monitoring, benchmarking the results, the cities are ranked by their achievements (many indicators) and timing kept. POINTER produces full reports on cities use of CIVITAS Plus programme, implementation of measures and provides the European Commission with the overal evaluation of CIVITAS Plus projects.


Sound communication on the methodology, a very good training of evaluators (auditors in the QUEST case) is crucial, as this has to ensure objectivity and the evaluator/auditor has to set up the co-operation at the city level. In the POINTER situation, is this provided by the CIVITAS Plus contracts the cities have for implementing their measures, and by their project evaluation plans (PEPs) and Local Evaluation Plans (LEPs) they had to provide as deliverables, while in the QUEST case the auditors will have to explain perfectly the process of approved QUEST methodology, benefits and conditions of QUEST auditing and achieve the MoU signatures and smooth auditing. Motivating the cities will be the key, as the cities will not have a budget allocation, like in the CIVITAS Plus contracts. A lot can be done through good personal involvement of the auditor and good description of QUEST and its application. The cities should also get the knowledge on the tool, its recognition and possible use in the future.

POINTER


The total quality assessment should include personnel and staff allocation to sustainable mobility measures implementation, managing and evaluating SUMPs. The reality should be checked while the auditor visits the city, comparing the statements in city plans, reports to measures implemented and staff working on the SUMP topics.

Reviewed by Radomira Jordova, CDV


Full project name: ECOMOBILITY SHIFT Short name: SHIFT

http://www.ecomobility.org/shift-project

(accessed 20 Aug 2011).

2010-13 √√√ Project is very similar in terms

of its aims to (i) audit and certify cities‟ transport policies by means

http://www.ecomobility.org/shift-project


03/01/2012 167 TTR

of TQM.


IEE-STEER ICLEI - Local Governments for Sustainability, Bonn, Germany; Napier University Edinburgh, United Kingdom;

Mobiel 21, Leuven, Belgium; Traject, Brussels, Belgium; Trivector, Lund, Sweden; Interface for Cycling Expertise (I-CE), Utrecht, The Netherlands; Municipality of Burgas, Bulgaria; Municipality of Miskolc, Hungary

Remit

SHIFT

Remit

The ECOMOBILITY SHIFT project aims to “develop a method to assess, improve and promote the environmental sustainability of local governments‟ transport and land-use planning policies”. A total quality management system for the auditing and certification of cities‟ policymaking processes and practices will be developed, with a focus on achievements in and progress towards a shift to „eco-mobile‟ behaviour: reduced travel, more walking and cycling, and greater public transport and shared car use. SHIFT is the first transport such transport sector project to combine QMS and labelling. It is intended that the SHIFT manual produced at the end of the project will become the main tool for improving mobility policies in cities.

It is also intended that SHIFT will be used to compare cities by means of benchmarking indicators, this serving as an incentive for municipalities to improve outputs, results and the methods used to deiver them.

The SHIFT tool will be used in 5 pilot cities as part of the project: Burgas (Bulgaria), the East Dunbartonshire region (UK), Miskolc (Hungary), Lund (Sweden) and Turnhout (Belgium).

The project has the following workplan:

1. Definition of the ECOMOBILITY labeling scheme

2. Definition of ECOMOBILITY criteria and Quality Management System

3. Set up of auditing system

4. Pilot of the Labeling Scheme and refinement of criteria and auditing system

5. Consultation and Feedback.

Existing labelling and certification schemes have been reviewed as part of steps (1) and (2), and a technical document on suggested indicators and labelling criteria followed as part of stage (2). As of June 2011 the SHIFT criteria (indicators and their levels for both self-assessment and external auditing) were still being finalised, together with details of the labelling system, following considerable input from partners, external experts and participating cities as to their expectations/needs.


03/01/2012 168 TTR

Two newsletters have been produced so far:

- Issue 1 (December 2010), introducing the project: http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2010-11-23_-_SHIFT_Newsletter_Nov2010_final.pdf (accessed 20 August 2011)

- Issue 2 (April 2011), summarising the „quality circle‟ for the assessment and improvement of mobility management policies, the project‟s first workshop with external experts and a review of existing schemes: http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2011-03-01_-_SHIFT_Newsletter_March_2011.pdf (accessed 20 August 2011).

Description of auditing process Not applicable.

Reach/implementation Not applicable.

Role of the auditor Not applicable.

Label or certification Not applicable.


See below.


See review of SHIFT deliverable D3.1 below.

Reviewed by Simon Field, EURIST


ECOMOBILITY SHIFT Deliverable D3.1: Review of Existing Quality Management and Labelling Schemes

ECOMOBILITY SHIFT Deliverable D3.1: Review of Existing Quality Management and

Labelling Schemes http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP3/D_3.1_-_Review_of_Existing_QMS_labeling_systems.pdf (accessed 20 Aug 2011).

Published 6 Jan 2011.

Last updated 31 March 2011.

√√√

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2010-11-23_-_SHIFT_Newsletter_Nov2010_final.pdf

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2010-11-23_-_SHIFT_Newsletter_Nov2010_final.pdf

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2011-03-01_-_SHIFT_Newsletter_March_2011.pdf

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/Newsletter/2011-03-01_-_SHIFT_Newsletter_March_2011.pdf


03/01/2012 169 TTR


IEE-STEER Napier University Edinburgh, United Kingdom; Mobiel 21, Leuven, Belgium; Traject, Brussels, Belgium

Remit The aim of this report was to understand how SHIFT can learn from other auditing schemes within and outside the transport sector, so as to adopt what works well, avoid duplication and avoid repeating mistakes. It posed the questions:

1. How and how well do the schemes work? 2. How and how well are they used in practice? 3. Are there schemes that “cover process, outputs and/or outcomes; allow user (citizen) input into the assessment of

cities‟ quality, allow comparison/benchmarking between different cities across Europe; and award labels to recognise achievement […] in process, outputs and/or outcomes”?

25 existing labelling and QMS schemes were analysed (see page 10), based on four main research topics:

1. benchmarking and sustainability indicators 2. quality management 3. auditing 4. labelling.

Description of auditing process This is not finalised, but is likely to be a hybrid of an adapted and expanded TQM system derived from (developed for auditing mobility management), and the enabler-/upstream process-related criteria from the CAF model and the Swedish Traffic Safety Audit.

The QMS will strongly resemble the quality circle, modified for the broader „ECOMOBILITY‟ criteria to be used in SHIFT.

SHIFT Deliverable D3.1

Reach/implementation

SHIFT is intended to be a universally applicable scheme in cities of varying sizes right across Europe. However, in practical terms it may be that the scheme is best applied in medium-sized monocentric cities (it would appear to be difficult to compare the polycentric region of East Dunbartonshire with other cities).

Role of the auditor Common elements of the SHIFT and QUEST approaches are as follows:

1. Auditors are external (it is not yet known how SHIFT will recruit them) 2. Auditors are expected to have considerable existing knowledge and expertise in sustainable transport (SHIFT does not

yet have criteria for this) 3. Auditors will be trained specifically and accredited in the use of the developed tool.

SHIFT auditors will undertake the assessment of cities‟ processes, outputs and outcomes using a checklist of indicators weighted/specific to the type/starting point of city (which itself is largely determined by self-assessment). They will be required


03/01/2012 170 TTR

to exercise judgment when assessing cities‟ progress towards EcoMobility, especially if the final QMS uses a large number of qualitative or otherwise subjective indicators.

uses three auditors per city, as the use of one is regarded as insufficient, possibly as a result of the use of subjective qualitative indicators.

Label or certification The review found that all schemes incorporate a form of certification.or grant permission for a logo e.g. ISO to be used.

In essence, labelling and certification is based on exceeding minimum scores for achievement levels (e.g. gold, silver, bronze proposed for SHIFT) against a range of indicators. In SHIFT the scores from the three areas of processes, outputs and outcomes are used to inform the overall score and label. For example, a city cannot achieve the highest label if deficient in one of the three.

SHIFT is currently considering whether to pursue a 3-level (gold, silver, bronze) labelling system or the 2-point scale used by the Swiss Energy City scheme.

It is proposed that labels remain valid for no more then 5 years, with shorter periods more appropriate for „starter‟-type cities. Validity should be determined by the auditor, also taking into account the time and cost commitment of cities.



It was concluded that , BYPAD, CAF, Siemens Complete Mobility Index, the Swiss Energy City scheme (Energiestadt) and the Swedish traffic safety audit are of most relevance to SHIFT, and therefore also of interest for QUEST.

Of these, the Swiss Energy City scheme has:

1. combined TQM and labelling 2. assessed processes and other enablers (such as governance structures), outputs (e.g. public transport service

quality) and outcomes (e.g. private vehicle km travelled per capita) 3. produced lists of threshold scores (minimum achievements) per indicator, required for (a) benchmarking in self-

assessment and auditing,(b) the awarding of labels based on relative performance 4. and could potentially be used outside Switzerland. This is a useful template for QUEST.

Schemes should not be too time-consuming, complex or expensive: this is a delicate balancing act between effectiveness and credibility vs ease of use and providing what cities want. SHIFT has engaged with cities and external experts to avoid


03/01/2012 171 TTR

developing an unattractive product.

Will one auditor per city be sufficient if QUEST relies heavily on qualitative and subjective indicators? This problem can be minimised by using quantitative data, especially independently verified statistics, to the greatest extent possible.

The credibility of QUEST would be maximised by attaining e.g. ISO9000 accreditation and being supported by at least one accreditation body.

QUEST is probably correct to focus on medium-sized cities rather than claiming to be applicable universally, and like SHIFT it should determine whether polycentric regions and clusters of towns can be dealt with as such (rather than assessing each small town or village separately).

Although benchmarking is not a stated objective of QUEST, note that it is necessary in order to determine the thresholds for labelling cities e.g. gold, silver, bronze; from 1 to 10, etc, as these are relative to average and best performance in class. Quantitative, objective indicators are preferable to those open to subjective interpretation.

The proposed SHIFT Network is analagous to the QUEST Academy, although the former is expected to be more involved in assisting cities with their self-assessment and the subsequent creation of improvement plans prior to full auditing. This is an area for QUEST to consider in more detail.

It is vital to think about marketability and post-project continuation strategies at an early stage, something that QUEST is doing! Of the schemes reviewed, only BYPAD has experienced significant uptake (~100 cities have been audited at least once); others have been little used. Although 2,100 cities are registered for CAF, it is not known how many of these actually use it. Incentives for cities to join are required: the Swedish Road Safety Audit was adopted widely because participating cities became eligible for additional transport funding. To an extent the success of QUEST will depend on the EC‟s financial incentives for SUMPs.



The most important lessons from the review of other schemes are listed above. Annex I of the document describes each of the 25 schemes in more detail, which can be revisited when reviewing QUEST‟s D3.1 report as necessary.



ECOMOBILITY SHIFT report: An ECOMOBILITY certification system for

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/2011-05-11_-

Published 11 May 2011. √√√ Vital for development of indicators for (i) participation in the

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/2011-05-11_-_Technical_brochure_LR.pdf

http://www.ecomobility.org/fileadmin/template/project_templates/ecomobility/files/SHIFT/WP7/2011-05-11_-_Technical_brochure_LR.pdf


03/01/2012 172 TTR

cities - Technical brochure _Technical_brochure_LR.pdf

(accessed 20 Aug 2011).

See also the following draft list of indicators for auditing in Google Docs redirected from:

http://www.ecomobility.org/shift/indicators (accessed 20 Aug 2011).

project and/or (ii) city self-assessment and external auditing.


IEE-STEER ICLEI; I-CE.

Remit The document introduces the project and summarises the detailed review of existing labelling and auditing schemes (see above). It then sets out details of the proposed assessment framework and labelling system, before explaining how the city audits might work in practice.

SHIFT report: An ECOMOBILITY certification system for cities - Technical brochure


See page 19 of document, but note that detailed audit procedures for SHIFT have not yet been published. The aim is to produce an audit report that identifies existing strengths and weaknesses in policymaking processes and achievements on the ground, the proposed label to be awarded and/or a list of recommendations for improvement.

It is assumed that cities will undertake self-assessment before applying to be audited externally. Cities unable to provide the minimum levels of information required for auditing will be asked to collect it (with assistance from SHIFT staff and the network of SHIFT cities as required) before reapplying for full SHIFT auditing at a later date.

The process is described as the way in which ECOMOBILITY performance is “systematically checked against a set of criteria” in a 4-step procedure carried out by specially trained external auditors:

1. Assessment of available information in the following areas:

http://www.ecomobility.org/shift/indicators


03/01/2012 173 TTR



;

2. Identification of gaps in the data and how these might be collected or alternative indicators selected 3. Data and policy document collection and analysis, plus field work and stakeholder consultation (politicians, planning

officers, user groups) 4. Audit report preparation.

Quantitative data are preferable, although it is likely that auditors will be faced with largely qualitative data, requiring skill and experience to form a judgment as to whether municipalities are performing well relative to other cities with similar profiles.


03/01/2012 174 TTR



Audit reports provide:

1. A city overview 2. A review of city performance for each indicator and overall scores for each of the categories of processes, outputs

and outcomes 3. The proposed label for the city 4. Recommendations for improvement in each of the categories listed in (2).

Reach/implementation Not yet implemented or available in a detailed manual.

Role of the auditor The auditor will undertake steps 1-4 above. It is proposed that SHIFT will deploy separate auditors and advisors: the latter will help cities to complete the self-assessment and other pre-requisites for auditing. SHIFT differs from QUEST in this respect.

Label or certification Only city governments/authorities will be eligible for labelling, and only following an external audit. Labels will be applied in recognition of achievements or improvements in (i) policymaking processes and other enablers, (ii) outputs (such as cycle infrastructure) and (iii) outcomes (such as modal split). It was originally suggested that there be a 3-point scale of gold, silver and bronze labels, but stakeholders have indicated a preference for a 2-level system: „EcoMobile city‟ and „gold‟. A final decision on this has not yet been made.

The 3-point labelling scale would be as follows (from page 18):


03/01/2012 175 TTR

- Gold: process (such as a QMS) in place and products and services being delivered and impacts actually achieved - Silver: process in place and products and services being delivered - Bronze: process in place.

The levels correspond to minimum aggregate indicator scores in each of the three areas (i)–(iii) above, with indicator weighting (relative importance) varying by city profile, e.g. size and starting position. Labels will be valid for a maximum of five years (corresponding to the longest electoral term in Europe), but the exact timeframe will be determined by the SHIFT auditor, based on starting position and ongoing commitment to EcoMobility.

It is also proposed to certify cities that have undertaken self-assessment and submitted an “internal audit annex improvement plan” as “SHIFT-users”. These cities would be eligible for tailored assistance (coaching) from a “SHIFT Network” of cities and advisors, as used in the previous project CAF. The purpose of this is to encourage the use of QMS and train cities‟ internal staff (city planning and transport officers) in the art of effective self-assessment. It is suggested that this progress and capacity-building could be recognised with a separate label.



Lessons learnt/conclusions for QUEST self assessment tool:

- the auditor, separate advisor and/or support network are likely to be required to provide assistance with quantitative data gathering

- offer a free-to-use web-based self-assessment questionnaire, with questions (a small subset of those used for full audits) designed to obtain the maximum amount of information in the lowest possible number of questions

- questions should cover processes/enablers, outputs and outcomes: assessing paper policies alone is insufficient.

Lessons learnt/conclusions for total quality management approach: NOTE: In SHIFT, TQM is the term used largely to refer to the use of indicators.

- TQM and associated indicators should cover processes/enablers, outputs and outcomes: assessing paper policies alone is insufficient

- Indicators should be as universally applicable as possible, but variation in wording and weightings is required to take into account different starting points while still facilitating meaningful comparisons and encouraging progress towards the next level

- Seek the views of the public and/or external experts when developing indicators - A degree of preliminary benchmarking is required, since some indicator scores are relative to average and best

performance in their respective areas - A support network is suggested as the means by which cities will be assisted to gather data and ultimately


03/01/2012 176 TTR

incorporate QMSs into their own policymaking processes.

Lessons learnt/conclusions for labelling approach:

- Keep it simple: use no more than 3 label categories, all of which should be easily understandable; - Labels should be valid for no longer than the electoral cycle.


Nothing additional to report at this stage.



Flemish PAC Mobility Handbook with additional guidelines on the website of the Flemish Government explaining the Flemish Mobility Decree of 20.4.01 amended on 20.3.09 on Mobility

http://www.mobielvlaanderen.be 2001 -10 √


Authors: Flemish Administration

Stakeholders: Department of Traffic and Road, the Flemish Transportation Company (De Lijn), Provinces

Remit Guidelines on the Process and Content of a SUMP for Flemish Communes and conditions for funding the studies and implementation of measures.

Plans are made by the commune in consultation with all stakeholders

Description of auditing process Documents are presented for the Provincial Audit Committee (PAC). This control organism consists of an external independent auditor and the stakeholders. In the first phases the PAC checks if the right actions to analyse the mobility

http://www.mobielvlaanderen.be/


03/01/2012 177 TTR

problems and to define sustainable scenarios are planned. In the last phase the PAC controls whether the “Policy Plan” :

- is structured in the required way e.g. in an information part and a directive part - consist of all elements: proper objectives, measures on land-use development and a hierarchic road structure,

measures for all mode networks and accompanying measures, evaluation and monitoring, an action programme - was subject of a participatory process or a public hearing.

Reach/implementation If Flemish cities and municipalities choose to develop or update a local mobility plan, they have to follow the procedure of where a quality control is done by the PAC. This procedure is foreseen in the Flemish decree on Mobility Covenants of 20.2.2001 and the Decree on Mobility Policy of 20.3.2009. In the near future (October 2011) a new Decree on Mobility Policy will make the development of local mobility plans mandatory with a legal basis.

Role of the auditor The external auditor chairs the PAC and checks whether the procedures are followed in the right way, if all required themes are discussed in the plan and if the participation process is ok. He writes the audit report.

Label or certification If all stages are done in the required way and the contents of the plan is accepted the Mobility Plan is “declared in conformity”.

Lessons learnt regarding development QUEST tool

The audit process focuses only on the procedure followed, the existence of a participation process and whether all themes are discussed. In principle no check of the quality and effectiveness of the actions is done. The ex-post evaluation of the actions (foreseen in the Monitoring and evaluation chapter) of the plan is not a solution for this.

Flemish PAC Mobility Handbook


QUEST should develop a way to check whether the actions and strategies defined in the Mobility Plan are taking on the right aspects to influence mobility in the required direction, e.g. are the right principles used ? e.g. the STOP principle : first walking, cycling, public transport and then the private car

Reviewed by Dirk Engel, Tritel


Max – Successful Travel Awareness Campaigns & Mobility Management Strategies

http://www.max-success.eu/ d.d. 2011-08-04 2006-9 √


EU, Sixth Framework Programme for 28 partners: Austrian Mobility Research, FGM-AMOR (project leader) (Austria); Mobiel 21 (Belgium); ILS Institut für Landes-

http://www.max-success.eu/


03/01/2012 178 TTR

Research and Technological Development (FP6)

und Stadtentwicklungsforschung gGmbH (Germany); Eric N. Schreffler, Transportation Consultant (USA); Equipo de Tecnicos en Transporte y Territorio, ETT (Spain); FIT Consulting (Italy); Lyle Bailie International Limited (UK); synergo (Switzerland); Timo Finke Consult Aachen (Germany); Traject (Belgium); Austrian Standards Institute (Austria); Trivector (Sweden); University of Piraeus Research Centre (Greece); University of Maribor, Faculty of Civil Engineering (Slovenia); Cracow University of Technology (Poland); Aristotle University of Thessaloniki (Greece); University of Lyon – CNRS-LET (France); Edinburgh Napier University (UK): University of Central Lancashire (UK); Vilnius Gedimas Technical University (Lithuania); University of Giessen, Institute for Applied and Empirical Social Research (Germany),#; Otto-von-Guericke-University of Magdeburg (Germany); Almada Municipal Energy Agency (AGENEAL) (Portugal); Almada Municipality (Portugal); Lazio Transport Company COTRAL (Italy); Kortrijk Municipality (Belgium); Tallinn Municipality (Estonia); Munich Municipality (Germany).

Remit MAX produced new research results – but also a whole array of tools: - Quality Management Approach for MM in Medium and Small Size Cities developed the tool - Integrating Mobility Management and land use planning developed the tool MaxLupo - Models of Behaviour Change and Prospective Assessment Tools developed the tools MaxExplorer, MaxSumo, MaxEva and

MaxSem - Successful Travel Awareness Campaigns & mobility Management Strategies: developed the tools MaxTag.

There are two relevant tool and Max Sumo. assesses the process Quality. MaxSumo is a method to develop an evaluation plan for projects.

MAX


Not applicable.





See below.


See reviews below.



03/01/2012 179 TTR



d.d. 2011-08-04

2006-9 √√√

European Committee For Standardisation (CEN) 2009: Code of practice for implementing quality in mobility management in small and medium sized cities

http://www.epomm.eu/docs/1055/CWA.pdf

d.d. 2011-08-04

October 2009

MAXimise Mobility Management

A guide to results from the MAX-project – aiming to extend,

standardise and improve Mobility Management

http://www.epomm.eu/index.phtml?ID=2182&id=2185

d.d. 2011-08-04

2009

Funding programme

Partners and other stakeholders involved/Authors

EU, FP6

Remit Focuses on small and midsized cities (defined 20.000-200.000 inhabitants), but can be implemented anywhere.

Quality management system for mobility management (QMSMM). Adopting a systematic approach to MM:

- ensures that MM-policy and MM-measure(s) are clearly defined, understood and accepted by all stakeholders (local politicians, senior management, main internal and external private and public partners) and the target group or wider group of citizens

- enables MM objectives to be clearly defined and closely integrated to the objectives regarding (sustainable) mobility of the city or organisation and aligned to the organisation‟s core business

- allows responsibilities for different parts of the MM-policy to be understood, allocated and agreed on - allows to set up partnerships outside the own MM-organisation (e.g. with other departments, with public - transport providers, schools, companies) - promotes a logical approach to planning and encourages more accurate estimating (human resources,


http://www.epomm.eu/docs/1055/CWA.pdf

http://www.epomm.eu/index.phtml?ID=2182&id=2185


03/01/2012 180 TTR

- time and costs) - provides a consistent means by which monitoring and evaluation can be effected - reassures senior management and stakeholders of the integral MM-approach; a possible reward such as a label can

strengthen this.



Process focuses on developing, monitoring assessing and improving both the overall Mobility Management policy and separate MM measures. It involves four steps (policy, strategy, implementation and monitoring and evaluation) and twelve sub steps, which are structured in a quality circle: The Max project developed an audit procedure to support the implementation of . It places the status-quo of each of the 12 elements (quality criteria) of the quality circle on a development ladder (6 stairs), and provides guidance to make further improvements.


03/01/2012 181 TTR


p. 10


03/01/2012 182 TTR


p. 10

Depending on the ambitions of the cities (unclear how to measure this), the current status of the QMSMM, the level of political awareness and feasibility in financial terms Max Q developed 5 different assessment procedures.

1. self assessment 2. small internal audit 3. internal audit 4. External audit 5. Certification and Benchmarking

Self assessment: short structured questionnaire for quick scan on quality status of the city (24 questions referring to 12 elements of tool) working with 5 scales (not 6?)...to tease stakeholders to get into deeper process.

Small internal audit and internal audit: more extended but still without supervision.

External audit: uses external auditor or facilitator to add own expertise to overall validity of audit report. The task of the external auditor could be described as follows):

- collecting evidence regarding the 12 different elements and checklist issues through bilateral contacts with the MM co-ordinator and/or MM team and analysing documents and information of all procedures in place, reports, minutes, etc.

- meeting with the MM co-ordinator and the MM team to get their quality assessment ratings as a baseline - attending a number of additional meetings (collective or bilateral; on average five meetings) with the main

stakeholders in MM and ask for their assessment and input for refinement of improvement action proposals - a consensus meeting with MM co-ordinator and MM team


03/01/2012 183 TTR

- drafting the audit report - presentation of the audit report to the MM team and main local stakeholders in MM.

There is no specific description of the auditing process available.

Certification and benchmarking is mainly developed for cities which are on level 4 or higher on the development ladder. Processes by two external auditors plus internal auditor (process not clearly described).

As we understand this only cities which are high on the ladder of development, therefore have good quality management implemented are qualified to receive a certificate.

Reach/implementation Scheme is new and only implemented in a few cities. Unclear if cities catch up with the possibility of self-auditing.

Role of the auditor Depending on de degree of assessment Max Q works with just internal auditors, one external auditor or even two external auditor. (process with external auditors not clear).

As the Qualification of external auditors is a key factor concerning the audit outcome it is strongly suggested that when assessing the conformity of an organisation to this document the external auditor(s) have a comprehensive background in mobility management and formal training as an auditor e.g. according to EN ISO 19011.

Label or certification Only well developed cities target level 4= can participate in benchmarking and certification


- Approach covers wider field of mobility - Different tracks (depth) depending on the level of development or engagement of cities = interesting approach for

QUEST - As we always want to work with a guided self assessment. The approach of having an internal auditor is not

appropriate - Low level cities are just triggered to think about QM and come up with own strategies. QUEST wants to guide all

cities nethertheless regarding their level of development, but will come up with different ways of improvement programmes.


- Different tracks (depth) depending on the level of development or engagement of cities is interesting for QUEST. There we also want to develop three different approaches depending on the level of development

- Certifying process seen to be very time consuming with more than one auditor and is only appropriate for well developed cities. This seems inappropriate because every city can make progress and the cities which have not done much work on QM yet need external guidance the most.

- produces a shortlist of actions based on the description and rating of the 12 elements. This shortlist gives guidance of the content of a action plan which can be build up after participating in the audit. The shortlist does not tell the city how the city can achieve this. This is a crucial shortcoming of .



03/01/2012 184 TTR


MaxSumo Guidance on how to plan, monitor and evaluate mobility projects

http://www.epomm.eu/docs/1057/MaxSumo_english.pdf

d.d.2011-08-04

August 2009


EU, FP6 Trivector, ILS

Remit The MaxSumo process describes how to make a monitoring and evaluation plan – MEP – for the projects. When the MEP has been developed you have a clear plan for how to proceed with the project as well as how to perform

the monitoring and evaluation of the project. MaxSumo does not focus on strategic plans.

Description of auditing process Not applicable.





Not applicable.


Not applicable.


http://www.epomm.eu/docs/1057/MaxSumo_english.pdf


03/01/2012 185 TTR


Mediate - Methodology for describing the accessibility of transport in Europe

http://www.mediate-project.eu/

d.d. 2011-08-02

http://www.aptie.eu/

d.d. 2011-08-03

Mediate newsletter 2011-01 http://www.aptie.eu/site/fileadmin/Mediate_deliverables/Mediate_Newsletter_4_Final.pdf.

Self Assessment Tool Manual (2010). Not found on projects website. Available from project partner.

2008-10 √√√


EC, Seventh Framework Programme for Research and Technological Development (FP7)

SINTEF (co-ordinator); AGE; Imob; Polis; Timenco; TIS.pt; Transport for London; TTR

Remit Mediate aimed to establish a common European methodology for measuring accessibility. Mediate developed a self-assessment tool that can be used by policy makers to identify areas where upgrading will improve the accessibility of public transport in their city.

Within the project a self assessment tool and a set of indicator for accessibility of public transport have been developed. A major output is the Good Practice Guide on accessibility of urban public transport systems. Good practice examples have been collected and analysed, with contributions from a range of cities and transport authorities/operators across Europe. They show how to apply a strategic approach to developing, implementing and reviewing accessibility solutions.

http://www.mediate-project.eu/

http://www.aptie.eu/

http://www.aptie.eu/site/fileadmin/Mediate_deliverables/Mediate_Newsletter_4_Final.pdf

http://www.aptie.eu/site/fileadmin/Mediate_deliverables/Mediate_Newsletter_4_Final.pdf


03/01/2012 186 TTR

Mediate


The self assessment tool is meant to help local stakeholders to evaluate to actual state of accessibility themselves. By involving local stakeholders in the assessment and the improvement plan, it is more likely that improvements will be put on the agenda and really be implemented. Moreover, there is a bigger chance that the proposed changes are regarded as important and appreciated by the end users. This involvement of stakeholders is a crucial aspect of TQM, in contrast with regular auditing

Evaluation process: The tool covers three phases 1. Initiative 2 Self assessment process and 3 Follow up (table).

Phase 1: Initiative

All partners are treated equally, so all of them can come up with the initiative to start with the evaluation process.


03/01/2012 187 TTR

Mediate


INITIATIVE – by user, politician, official or operator

1

2

3

SELF ASSESSMENT PROCESS

Meeting 1: Joint Assessment by evaluation group

& consensus on quality level of accessibility

Meeting 2: Agreement on objectives and measures

Assessment Report and Quality Plan

Individual Assessment by the evaluation group

MO

DE

RA

TO

R

Choice of

Moderator

users politicians officials operators

Creation of the evaluation group

FOLLOW UP

Integration of results in regular policy and advices

users politicians officials operators

p. 31

Moderator: guide participants through process. He should be independent e.g. playing no role in the process that is assessed. Moderator has qualification in quality management and expertise in public transport policy with a focus on accessibility issues. The moderator:


03/01/2012 188 TTR

Mediate


- is there to get the process started - assists the „initiator‟ (operator, city council, etc) to set up the evaluation group - collects relevant information - makes sure that the right questions are being answered - has experience in leading group sessions and facilitating discussions towards consensus forming - makes sure that every participant can play an active role in the meetings - is alert to opinions that are contrary to the fact finding and discrepancies are discussed and clarified. - if evaluation group is not able to suggest new accessibility measure, the moderator suggests actions based on the

results of the assessment and experiences in other cities - it is convenient to have someone assisting the moderator (no further specification).

Phase 2: Self assessment process

Ongoing process with fact finding and self assessment at certain moments, and each with their own use and function within the process of improving accessibility.

Kick off meeting to explain idea of self assessment and to agree om common goals of the process.

Fact finding responsibility of transport authority to feed moderator with relevant facts (background information like bus fleet, personal means, budget etc...).

For the fact finding, a set of indicators developed. Following these indicators, the moderator and the city administration gather the available information on accessibility. The list of indicators makes clear that accessibility has many aspects to consider and it is not likely that a city has all the information and data available. But the amount of information available is also an indication of how well developed the policy is.

Key indicators: (indicators are specified in WP2 deliverable: http://www.aptie.eu/site/fileadmin/Mediate_deliverables/MEDIATE_D22_Indicators_for_accessibility_of_urban_public_transport_final.pdf.

http://www.aptie.eu/site/fileadmin/Mediate_deliverables/MEDIATE_D22_Indicators_for_accessibility_of_urban_public_transport_final.pdf

http://www.aptie.eu/site/fileadmin/Mediate_deliverables/MEDIATE_D22_Indicators_for_accessibility_of_urban_public_transport_final.pdf


03/01/2012 189 TTR

Mediate


No Key indicator Measure / scale (most positive alternative first)

Policy indicators

Policy and investment

A1 Accessibility plan: Current plan at urban level. Yes / no

A2 End-user involvement (of older and disabled people) in planning,

implementation, monitoring and evaluation. Qualitative description

A3 Integrated accessibility policy Qualitative description

Service operations and standards

B1 Meeting user needs (staff, personal security measures, complaint

procedures, and feedback) Qualitative description

B2 Accessibility maintenance: Plan, routines, and monitoring. Qualitative description

B3 Fare policies & alternative services Qualitative description

Performance indicators

Information and ticketing systems

C1 Accessible information Policy approach: Integrated / system-oriented / isolated / ad hoc / none

C2 Accessible ticketing (buying & validating) Policy approach: Integrated / system-oriented / isolated / ad hoc / none

Vehicles and built environment

D1

Accessible vehicles and built environment (pedestrian environment, stops

and stations, platforms and vehicles) Policy approach: Integrated / system-oriented / isolated / ad hoc / none

Seamless travel

E1 Seamless travel (modal interoperability) Policy approach: Integrated / system-oriented / isolated / ad hoc / none


03/01/2012 190 TTR

Mediate


Some weeks before

Meeting with all parties

involved in the

assessment

What is SA?

Exercise LoD

Fact finding

Data collection

Individual assessments

Individual assessmentsAssessment per

Member Evaluation

Group

Some weeks before

Preparatory

meeting

Data analysis

Week before

Merge results for

M1, M2, M..., M10

SA Meeting 2

Objectives and

measures

Action plan

SA Meeting 1

Joint assessment

current situation

Meeting

Fact finding &

Technical visit

Conclusions

Assessment & Fact

finding

Assessment

Day 1

Day 2

Conclusions of Joint

Assessment per

Module

Action plan

Some weeks after

Result meeting

SWOT

Actions

...

Meeting with evaluation group

Responsibility for members evaluation group

Responsibility for the moderator

Creation of the evaluation group: users, politicians, officials, operators, ...

Follow up

integration of results in regular policy and advices

§ Public transport Authority

§ Operators

§ Road Authority

§ End-users

Responsibility for the Lead Partner of the evaluation

The different tasks in the process:

Self Assessment Tool for the Accessibility of Public Transport

September 2010

1

2

3

p. 38


03/01/2012 191 TTR

Mediate


Analysis and scoring: After the first meeting of the evaluation groups, the scores of all modules can be presented. An appropriate presentation can be given with the so called radar plot, or spider‟s web chart.

p.41

Scores are given per module. This is actually were the analysis stops. An overall score of the accessibility will not be calculated. The purpose of the assessment is to describe the accessibility in its various aspects and to development a plan for improvement of the system.


03/01/2012 192 TTR

Mediate


Action plan: The analysis and scores for the different modules give indications where to find the best opportunities for improvements. During the second meeting suggestions are put forward to make improvements on the weakest modules. The result of the second meeting is an outline for an improvement programme. In the Assessment Report, the moderator will give recommendations for focusing on the next improvements. The assessment report is presented to the city authority. Responsible parties will be pointed out (city authority, road managers, transport operator), objectives, milestones and targets formulated and an indication is given for the required budget.

Phase 3: Follow up

The results from the Assessment Report and Quality Plan have to be translated into policy of the different organisations that have been represented. Local governments, public transport authorities, operators should translate the outcomes into short and long term policy goals together with budgets and people who are responsible. For user organisations the situation might be special as they have a different position. As they are often one of the formal partners in the development and assessment of public transport policy, they can use the Assessment Report and the Quality Plan to set goals for advising the different authorities responsible for public transport.

Reach/implementation Toot tested on two sites (Flanders region, de Lijn and City of Lisbon).

Role of the auditor External expert (moderator), no auditor.

Label or certification None.


- For reasons of efficiency and effectiveness the evaluation group shouldn‟t be too big, preferable having fewer than 10 members.

- Formal agreement of participants can be helpful 9signed by responsible person). In cases where some parties are not very eager to participate or to contribute with information, such an agreement can be useful to show that the assessment is considered to be important and that it has been agreed on at high level.

- In mediate is the choice of the geographical area to be assessed one of the crucial steps. For smalls scales the assessment can address improvement for one particular bus line which connects two municipalities. Over the assessment for very large geographical area‟s, which will be addressed, which the QUEST approach) Mediate refers: that the assessment will be at a more general level. Detailed information on conditions of particular stations are not available and end users are not likely to know all detailed information.

Method:

- scores per module and not an overall score of the accessibility is produced in Mediate. It is not a beauty contest but must show to participant weak modules so objectives and measures for improvement can be distilled.

- 2 meetings 1 analysing 2 action plan


03/01/2012 193 TTR

Mediate


Mediate has tested the approach on two sights the Flanders region/De Lijn and the city of Lisbon. Lesson learned from this:

- prepare to commit stakeholders: It takes a lot of time to indentified, address and invite stakeholders to participate in the Self-Assessment process. It might not be evident which organisation to address and which persons to contact. In many cases, organisation may need time to consider the invitation and to appoint the right delegate.

- lull introduction of process is very necessary: participants have encountered difficulties in filling out the forms or preparing for the meetings. Misunderstanding occurred on the status of the forms (not being an ordinary survey) and the discussions during the meeting (not being an official forum). Although these issues have been properly addressed during the meeting, the process needs improvement on the proper introduction to Total Quality Management and the status of the forms and the meetings.

- Self assessment is not a do it yourself approach, nor an audit. It is essential that the stakeholder evaluate the system themselves and discuss their differences of opinions. This is in contrast with an audit, where judgments will be given by an independent external auditor. Mediate strongly recommends that local stakeholders appoint an experienced moderator to support them and guide them through the assessment process. The moderator will not make any final judgments, that is what the stakeholders do themselves: collectively and in the well structured process.

By repeating the procedure at regular intervals, the indicators will suggest whether the development is sustainable, investments are well spent and contracts are followed up.


Not applicable.

Reviewed by Maja van der Voet, Ligtermoet and Partners


03/01/2012 194 TTR

Appendix 6

Full check-list of questions used by the BUSTRIP Project

1. Organisation of SUTP work

How is SUTP-work in your city organised?

• SUTP-working group established ?

• Several city departments are involved in the group ?

• Responsibility for SUTP-work defined ?

• SUTP-working group is meeting regularly ?

• Time and money for SUTP-work allocated ?

• City politicians are informed ?

• Political support for SUTP-work ?

• Long term structure for SUTP-work established ?

2. Stakeholder participation

To what extent, and how, are stakeholders involved in the SUTP-process?

Internal stakeholders

• Relevant stakeholders within the municipal organisation identified ?

• Defined how they affect or are affected by transport issues ?

• Are they informed about the municipal SUTP-work ?

• Are they involved in the SUTP-work ?

Official regional/national stakeholders (e.g. regional administration)

• Relevant official regional and national stakeholders identified ?

• Defined how they affect or are affected by municipal transport issues ?

• Are they informed about the municipal SUTP work ?

• Are they involved in the SUTP-work ?

External stakeholders (outside the administration)

• Relevant citizen/private stakeholder groups identified ?

• Defined how they affect or are affected by municipal transport issues ?

• Informed them about the municipal SUTP work ?

• Are they involved in the SUTP work ?

• Has the SUTP been presented to and discussed by different local political parties ?


03/01/2012 195 TTR

3. Coordination of policies and plans

How well are existing policies and plans affecting urban transport coordinated ?

• Policies and rules that affect transport identified and listed ?

• Checked for contradictions and support for sustainable urban transport ?

• Policies and rules that conflict with goals of sustainable urban transport listed ?

• Issue of conflicting goals raised to responsible decision makers ?

• Conflicting rules and policies changed ?

• Regional and national goals that are affected by the cities transport identified ?

• Regional and national rules that affect the cities transport issues identified ?

• Regional and national rules that conflict with goals of sustainable urban transport listed ?

• Issue of conflicting goals raised to responsible decision makers ?

4. Capacity building

Has the city administration the necessary capacity and know-how to successfully address sustainable urban transport issues ?

• Sufficient manpower to work with SUTP-issues ?

• Sufficient know-how to work with SUTP-issues ?

• Learning opportunities and experience exchange offered to staff ?

• Politicians involved in capacity building on SUTP-issues ?

• Systematic approach to increase knowledge and experience on sust. transport issues within the organisation (e.g. by pilot projects that can be scaled up) ?

5. Drivers

Have the drivers (factors) that influence traffic development been identified and trends understood?

• Drivers for transport development in the city identified ?

• Identified what and who influences them ?

• Development trends for each driver listed ?

• Positive drivers listed, analysis of possible supporting actions performed ?

• Negative drivers listed, analysis of possible actions to influence drivers performed ?

6. Analysis of current situation

How well does the city understand its current transport situation and is there reliable data?

• Statistics and regular measurements on traffic development (general) ?

• Statistics and regular measurements on car transport ?

• Statistics and regular measurements on public transport ?

• Statistics and regular measurements on freight transport ?


03/01/2012 196 TTR

• Measurement system, routines and statistics on noise and air pollution ?

• Statistics and follow up tools on road safety ?

• Maps that locate the distribution of noise, air pollution and accidents over the city ?

• Current national and EU pollution limits fulfilled ?

7. Gender equity and equality

Are gender and equality issues sufficiently considered within your SUTP work?

• Is there a balanced gender representation in the SUTP working group ?

• Is there a balanced gender representation amongst decision makers concerning traffic issues ?

• Are the specific needs of women considered in planning and the decision making process (gender mainstreaming) ?

• Is equal accessibility for all groups considered in planning and decision making (equity, e.g. for the disabled, elderly or parents) ?

8. SUTP scope and definition

Is there a clear scope and definition of the fields the SUTP covers?

• Is the scope/goal of the SUTP clearly defined ?

• Is it clearly defined what topics and development fields the SUTP should cover ?

• Is it transparent what other policies and plans are part of or linked to the SUTP ?

9. Analysis of baseline scenario

Is there a reliable and useful baseline scenario against which the changes in the city can be compared ?

• Has the city a set of measurable indicators that can be used to describe the traffic situation and traffic impacts in the city ?

• Are reliable measurement values available for all indicators ?

• Is the measurement methods sufficiently reliable to be used for comparisons over several years ?

• Has a baseline year against which traffic development is compared been chosen ?

10. Vision, objectives and targets

A clear vision of sustainable urban transport defines the direction the transport development needs to take. Objectives and targets that should be reached at a certain year define the goals in a nearer future.

• Has the city a defined vision how transport/traffic in the city should develop ?

• Is this vision compatible with other outspoken development visions and objectives ?

• Have the objectives with the SUTP been clearly defined ?


03/01/2012 197 TTR

• Are there specific objectives for personal transport ?

• Are there specific objectives for freight tranpsort ?

• Have target values been defined for all of the citiy‟s transport indicators ?

• Has a timeplan when the target values should be reached been decided ?

11. Implementing action, allocating finance

Is there evidence that there is enough planned action to be able to reach the cities targets?

• Has a list of planned SUTP-actions been defined ?

• Has the expected effect of the planned actions been assessed ?

• Are the actions sufficient to reach the city„s targets ?

• Do all SUTP targets have actions defined to reach them ?

• Can the planned actions be scaled up to city level?

• Is there a time plan for the suggested actions ?

• Have sufficient funds been allocated for the suggested actions ?

12. Policies and measures

Have policies relevant for SUTP been defined and necessary measures been taken ? Have other city policies been linked to and coordinated with the SUTP ?

• Existing policies and measures that affect transport are listed in and linked to the SUTP ?

• There are clear references to existing plans and policies in the SUTP ?

• The interconnection and hierarchy of transport related policies has been defined in the SUTP ?

• All plans and measures that can affect transport are checked against whether they are supporting the SUTP goals ?

• A set of measures to move towards the SUTP vision has been defined ?

13. General principles

• A set of general principles that support the SUTP-goals has been defined in the SUTP (e.g. planning for reduced transport need, equal right to accessibility for all groups of society) ?

• The city„s planning documents have been checked against these principles and if necessary, changes haven been made (Agenda21, physical planning..) ?

14. Reducing the need for transport

• Does the SUTP contain the principle of reducing the need for transport ?

• Are there goals, measures and targets to reduce the need for personal transport ?

• Are physical planning principles in the city adapted to reduce the future need for personal transport ?


03/01/2012 198 TTR

• Are there goals, measures and targets to reduce the need for goods transport ?

• Are physical planning principles in the city adapted to reduce the future need for goods transport ?

15. Transport management

• Does the SUTP contain goals, measures and targets for transport management ?

• Does the SUTP contain goals, measures and targets for improving public transport ?

• Does the SUTP contain goals, measures and targets for parking management ?

• Does the SUTP contain goals, measures and targets for congestion management ?

• Does the SUTP contain goals, measures and targets for freight transport management ?

16. Clean and silent transport system

• Does the SUTP contain goals, measures and targets for air quality improvements ?

• Does the SUTP contain goals, measures and targets for reduction of greenhouse gases (mainly CO2) from transport ?

• Does the SUTP contain goals, measures and targets to promote sustainable modes of travel i.e biking and walking ?

• Does the SUTP contain goals, measures and targets to promote sustainable modes of transport i.e rail ?

• Does the SUTP contain goals, measures and targets to promote clean vehicle technology ?

• Does the SUTP contain goals to reduce noise impact in the city ?

17. An accessible and safe transport system

• Does the SUTP contain goals, measures and targets for accessibility for different population groups (e.g. inhabitants without cars, children and impaired people) ?

• Does the SUTP contain goals, measures and targets for traffic safety ?

• Does the SUTP contain goals, measures and targets for traffic safety explicitly for children ?


• Are responsibilities clearly assigned in the SUTP ?

• Has each proposed measure been assigned to a responsible unit / person ?

• Have resources been assigned in the city/department budgets so that the planned measures can be realised ?

• Has it been defined what actions should be taken if defined targets are not reached ?


03/01/2012 199 TTR

19. Plan adoption, approval and assessment

• Is it clearly defined on what level/by whom the SUTP needs to be adopted and approved to be valid (e.g. the city council) ?

• Has the SUTP been presented to and discussed by different political parties ?

• Has the SUTP been politically approved ?


• Is it defined how progress towards SUTP goals will be monitored ?

• Are there set dates for evaluation of progress of the SUTP work ?

• Is there a clearly defined unit/person that is responsible to evaluate the progress of SUTP work against the defined goals and targets ?

• Has that unit/person the necessary competence, tools and resources allocated for the evaluation of the progress towards SUTP targets ?

• Is there a time plan for evaluation of SUTP progress ?

• Is it defined to whom the evaluators will report their results ?