annex c: statistical assessment of the cultural and creative...

Annex C: Statistical Assessment of the Cultural and Creative Cities Index 2019 | 1

Annex C: Statistical Assessment of the Cultural and Creative Cities Index 2019

The Cultural and Creative Cities Monitor (hereinafter ‘the Monitor’) aims to capture and express complex and indeterminate concepts related to the ‘Cultural Vibrancy’, ‘Creative Economy’ and ‘Enabling Environment’ of cities in Europe. Similarly to the statistical annex accompanying the first edition of the Monitor in 2017, the present statistical annex analyses and comments on the methodological choices made in the selection of the 29 indicators and their organisation into nine dimensions, three sub-indices and an overall index (the The Cultural and Creative Cities (C3) Index). The analysis was performed and is reported here in order to maximise the reliability and transparency of the model (the C3 model). It should enable users to draw more relevant, meaningful and useful conclusions from the results presented in the main report and in the online version of the Monitor1.

As in the previous statistical assessment, the present assessment of the C3 Index 2019 focuses on two main aspects: the statistical coherence of the structure; and the impact of key modelling assumptions on the C3 scores and ranks2.

This analysis complements the rankings in the C3 Index with confidence intervals, including for the three sub-indices (‘Cultural Vibrancy’, ‘Creative Economy’, and ‘Enabling Environment’), in order to test the robustness of these rankings to the computation methodology.

Overall, we found that the ranks are fairly robust to modelling assumptions, namely:

ĐĐ the C3 Index rank is close to the median rank (fewer than two positions away) for 75 % of the cities;

ĐĐ the ‘Cultural Vibrancy’, ‘Creative Economy’ and ‘Enabling Environment’ ranks are close to the median rank for 75 %, 73 % and 66 % of the cities, respectively.

These results suggest that the proposed measures provide a suitable summary for cities’ performance on culture and creativity.

(a) Statistical coherence in the cultural and creative cities framework

An initial assessment of the C3 Index was undertaken in 2017. No critical issues were identified in the 2017 model. The underlying concepts and framework used to describe Cultural and Creative Cities therefore remained essentially the same as those used for the C3 Index 2017.

The assessment of the statistical coherence of the C3 Index 2019 followed four main steps, described hereafter.

Step 1. Relevance

The same 29 variables as those used in 2017 were selected based on statistical coherence, country coverage and timeliness criteria. To take into account differences between cities, variables were scaled, either at source or by the Joint Research Centre (JRC) as appropriate and where needed. Most variables have been expressed in per capita terms (see Table A1 in the appendix for more details).

2 | Annex C: Statistical Assessment of the Cultural and Creative Cities Index 2019

Step 2. Data checks

The most recently released data were used for each city. The cut-off year was maintained at 2010. Cities were included if data availability was at least 45 % at the main index level and at least 33 % for the ‘Cultural Vibrancy’ and ‘Creative Economy’ sub-indices (see Table A2 in the appendix for more detail on cities’ data coverage). Although the data availability requirements were maintained at relatively low levels in order to allow more cities to be included in the analysis, data coverage has actually improved compared to 2017 for most cities, due to both the maintained 2010 cut-off and the use of new and more comprehensive data sources for some variables (see also technical annexes A and B for more details, available on the Cultural and Creative Cities Monitor Online). Most of the 179 cities included in the calculation of the C3 Index indeed have 75 % data coverage both at the main index and sub-index level (namely: 79 % of the selected cities have 75 % data coverage on the C3 Index, 66 % have 75 % data coverage on ‘Cultural Vibrancy’, 58 % have 75 % data coverage on ‘Creative Economy’, and 80 % have 75 % data coverage on ‘Enabling Environment’).

The dataset is characterised by satisfactory data coverage (81 % in a matrix of 29 variables × 190 cities). Missing data for each city were estimated using a refined and at the same time simplified two-step approach, compared to the previous edition3. In the first step, the national average is used for missing values in the perception-based variables related to foreigners and trust. This first step made it possible to fill in 37 % of the 1 064 values missing in the dataset. In a second step, the last remaining missing values were replaced with the k-nearest neighbor (k-NN) method using the average of the values of the three nearest (or statistically closest) neighbors. These were identified using the triplet population-gross domestic product (GDP)-employment rate, based on continuous values rather than the attributed group (i.e. categorical value from ‘1’ to ‘5’ for the five population-GDP-employment rate groups)4. This second step made it possible to fill in 63 % of the 1 064 values missing in the dataset.

Potentially problematic variables that could bias the overall results were identified as those having absolute skewness greater than 2 and kurtosis greater than 3.55, and were treated by winsorisation. A total of 51 values in 20 indicators were treated by winsorisation: values distorting the indicator distribution were assigned the next highest value, until skewness and/or kurtosis entered within the ranges specified above (see Table A3 in the appendix for more details).

Step 3. Statistical coherence

The statistical coherence of the Cultural and Creative Cities Monitor 2019 consists of a correlation and cross-correlation analysis to examine the structure of the data and the grouping of indicators into nine dimensions. It also includes a comparison of the expert-based weights assigned to the key components of the C3 Index (the nine dimensions and three sub-indices) with the respective implicit weights.

Correlation and cross-correlation analysis

Correlation analysis shows that within each dimension all correlations of the underlying indicators with the respective dimension are strong and positive (greater than 0.5 in all but two cases)6. Furthermore, a more detailed analysis of the cross-correlations of the indicators with the Monitor dimensions confirms the expectation that the indicators more strongly correlated with their own dimension than with any other (see Table 1), in all but two cases7.

These results suggest that the conceptual grouping of indicators into dimensions in the Monitor framework is statistically sound and that all the indicators influence the variation in the city scores in their respective dimension.

panelfr

Sticky Note

Marked set by panelfr


Table 1. Statistical coherence in the Cultural and Creative Cities framework: Pearson correlation coefficients between indicators and dimensions

Dimensions Indicators D1.1 D1.2 D2.1 D2.2 D2.3 D3.1 D3.2 D3.3 D3.4

D1.1 Cultural Venues & Facilities

Sights & landmarks 0.630 0.335 0.110 – 0.079 – 0.247 – 0.006 – 0.140 – 0.141 – 0.179

Museums & art galleries 0.750 0.487 0.252 – 0.002 – 0.131 0.117 0.002 – 0.070 0.043

Cinemas 0.753 0.270 0.299 0.075 – 0.171 0.105 – 0.127 – 0.014 – 0.021

Concert & music halls 0.694 0.349 0.250 0.079 0.045 0.378 0.179 0.128 0.117

Theatres 0.713 0.359 0.422 0.162 – 0.039 0.276 0.052 0.135 0.276

D1.2 Cultural Participation & Attractiveness

Tourist overnight stays 0.399 0.549 0.248 0.091 – 0.023 – 0.032 0.007 0.083 0.004

Museum visitors 0.407 0.713 0.332 0.221 0.005 0.052 0.082 0.153 0.097

Cinema attendance 0.388 0.385 0.326 0.076 0.020 0.318 0.020 0.094 0.161

Satisfaction with cultural facilities

– 0.036 0.544 0.371 0.436 0.306 0.184 0.352 0.454 0.531

D2.1 Creative & Knowledge-based Jobs

Jobs in arts, culture & entertainment

0.484 0.548 0.862 0.311 – 0.033 0.157 0.194 0.337 0.183

Jobs in media & communication

0.233 0.478 0.888 0.478 0.362 0.476 0.274 0.494 0.373

Jobs in other creative activities

0.292 0.538 0.879 0.430 0.284 0.328 0.291 0.577 0.416

D2.2 Intellectual Property & Innovation

ICT patent applications 0.069 0.343 0.353 0.877 0.090 0.256 0.259 0.320 0.453

Community design applications

0.079 0.343 0.427 0.795 0.189 0.213 0.160 0.387 0.366

D2.3 New Jobs in Creative Sectors

Jobs in new arts, culture & entertainment enterprises

– 0.185 0.032 0.016 – 0.062 0.721 – 0.098 0.086 0.100 0.026

Jobs in new media & communication enterprises

– 0.129 0.181 0.248 0.262 0.893 0.249 0.165 0.285 0.150

Jobs in new enterprises in other creative activities

– 0.028 0.178 0.236 0.124 0.907 0.117 – 0.002 0.298 0.086

D3.1 Human Capital & Education

Graduates in arts & humanities

0.237 0.122 0.164 0.056 – 0.054 0.597 – 0.055 0.044 0.073

Graduates in ICT 0.156 0.025 – 0.021 0.160 – 0.036 0.710 0.092 – 0.075 0.181

Average appearances in university rankings

0.179 0.368 0.552 0.303 0.323 0.654 0.198 0.525 0.228

D3.2 Openness, Tolerance & Trust

Foreign graduates 0.182 0.152 0.262 0.085 0.078 0.189 0.548 0.299 0.352

Foreign-born population 0.308 0.413 0.572 0.412 0.043 0.146 0.473 0.527 0.471

Tolerance of foreigners – 0.102 0.120 0.116 0.141 0.233 0.081 0.748 0.028 0.291

Integration of foreigners – 0.174 – 0.103 – 0.133 – 0.150 – 0.001 – 0.011 0.638 – 0.047 – 0.064

People trust – 0.062 0.240 0.207 0.346 – 0.005 0.081 0.752 0.351 0.682

D3.3 Local & International Connections

Accessibility to flights – 0.047 0.250 0.333 0.363 0.173 0.201 0.322 0.853 0.477

Accessibility by road 0.003 0.186 0.302 0.263 0.057 0.076 0.144 0.670 0.422

Accessibility by train 0.135 0.437 0.564 0.334 0.373 0.295 0.283 0.802 0.361

D3.4 Quality of Governance

Quality of governance 0.093 0.409 0.358 0.493 0.115 0.259 0.555 0.529 1.000


Importance of the dimensions and sub-indices in the Cultural and Creative Cities framework

The C3 Index is calculated as a weighted average of its three sub-indices (namely 40 % each for ‘Cultural Vibrancy’ and ‘Creative Economy’ and 20 % for ‘Enabling Environment’), whilst the three sub-indices are calculated as weighted averages of the respective underlying dimensions: the ‘Cultural Vibrancy’ sub-index is the weighted average of D1.1 Cultural Venues & Facilities (50 %) and D1.2 Cultural Participation & Attractiveness (50 %); the ‘Creative Economy’ sub-index is the weighted average of D2.1 Creative and Knowledge-based Jobs (40 %), 2.2 Intellectual Property & Innovation (20 %), and 2.3 New Jobs in Creative Sectors (40 %); the ‘Enabling Environment’ sub-index is the weighted average of D3.1 Human Capital & Education (40 %), D3.2 Openness, Tolerance & Trust (40 %), D3.3 Local & International Connections (15 %) and D3.4 Quality of Governance (5 %).

The weights for the C3 Index were elicited, using the budget allocation method8, by around fifteen international experts during the second participatory workshop of the Monitor in November 2016. While weights are often assigned to the components of an index to reflect the components’ effective importance in the index, in practice the data correlation structure and the data variances do not always allow the weights assigned to the variables to match their importance.

This section compares the expert-based weights assigned to the nine dimensions and the three sub-indices with their ‘implicit weights’. The implicit weights are calculated here with the squared Pearson correlation coefficient, otherwise known as the coefficient of determination. Table 2 shows that the coefficients of determination (‘importance’ measures) of the C3 Index components are in general similar to the expert-based weights. For example, city variations in scores on D1.1 Cultural Venues & Facilities can capture 53 % of the variance in the ‘Cultural Vibrancy’ scores, just slightly more than the 47 % captured by variations in D1.2 Cultural Participation & Attractiveness. Hence, the implicit weights for the two components of ‘Cultural Vibrancy’ are very similar to the 50-50 % weights assigned by the experts to these two dimensions. The most notable divergence between an expert-based weight and an implicit weight in the Monitor framework is observed for D3.4 Quality of Governance: despite the modest 5 % weight assigned by the experts, this dimension captures 24 % of the variation in city scores in the ‘Enabling Environment’ sub-index. This can be explained by the good correlation between D3.4 Quality of Governance and two of the dimensions in the Monitor framework, namely D3.2 Openness, Tolerance & Trust and D3.3 Local & International Connections (Pearson correlation coefficient slightly above 0.5). Nevertheless, within ‘Enabling Environment’, D3.4, Quality of Governance, is less important than D3.2, Openness, Tolerance & Trust, as foreseen by the experts. At the same time, however, D3.4 Quality of Governance has the same implicit weight as D3.1 Human Capital & Education, while experts proposed attributing greater importance to human capital over governance. At the main index level, ‘Cultural Vibrancy’ and ‘Creative Economy’ are more important than ‘Enabling Environment’, although ‘Creative Economy’ seems to capture somewhat more variation (42 %) than ‘Cultural Vibrancy’ (33 %), despite the expert-based weights putting these two sub-indices on a par.


Table 2. Expert-based weights and importance measures for the main components of the C3 Index

1. Cultural Vibrancy

2. Creative Economy

3. Enabling Environment

C3 Index Expert-based

weights

Implicit weights

(rescaled to sum to 100 %)


89 % 15 % 17 % 55 % 50 % 53 %


83 % 52 % 40 % 79 % 50 % 47 %


56 % 77 % 49 % 82 % 40 % 39 %


27 % 62 % 43 % 58 % 20 % 25 %


1 % 73 % 20 % 46 % 40 % 35 %


32 % 35 % 67 % 51 % 40 % 24 %


13 % 28 % 79 % 41 % 40 % 33 %


23 % 55 % 60 % 56 % 15 % 19 %


27 % 39 % 67 % 51 % 5 % 24 %

1. Cultural Vibrancy 100 % 37 % 31 % 76 % 40 % 33 %

2. Creative Economy 37 % 100 % 50 % 85 % 40 % 42 %


31 % 50 % 100 % 67 % 20 % 25 %

NB: The first four columns are the Pearson correlation coefficients. The ‘implicit weights’ are the squared Pearson correlation coefficients rescaled in order to sum to 100 %.

Step 4. Qualitative review

Finally, the C3 Index results, including city classifications and relative performances in terms of the three sub-indices and within the four population groups (9), were evaluated by the development team and the experts to verify that the overall results were consistent with current evidence, existing research and prevailing theory on culture and creativity.

Notwithstanding the positive outcomes of these statistical tests regarding the soundness of the C3 model, it is important to note that the C3 model can be expected to improve as better data become available, in particular on cultural venues and activities, cultural demand and funding for culture and creativity.


(b) Impact of modelling assumptions on the C3 Index results

Every score on the overall C3 Index and its three sub-indices depends on modelling choices. These choices were based on literature review (e.g. for the selection of variables), standard practice (e.g. min-max normalisation in the [0, 100] range) and expert opinion (e.g. for weights assigned to the nine dimensions and the three sub-indices), or were driven by statistical analysis (e.g. treatment of outliers and missing data estimation). The robustness analysis described hereafter is aimed at assessing the combined impact of key modelling choices on the city rankings. This uncertainty analysis is, to some extent, an explicit acknowledgement of and attempt to address the fact that the aggregate city scores are not calculated under conditions of certainty10.

Table 3. Uncertainty analysis for the C3 Index 2019: normalisation and weights

II. Uncertainty in the normalisation formula at the indicator level

Reference: min-max Alternative: percentile ranks

II. Uncertainty in the weights at the dimension level

Dimension/sub-index Reference value for the weight (within the sub-index)

Distribution assigned for robustness analysis (within the

sub-index)

D1.1 Cultural Venues & Facilities 0.5 U[0.38, 0.63]


0.5 U[0.38, 0.63]


0.4 U[0.3, 0.5]


0.2 U[0.15, 0.25]

D2.3 New Jobs in Creative Sectors 0.4 U[0.3, 0.5]

D3.1 Human Capital & Education 0.4 U[0.3, 0.5]

D3.2 Openness, Tolerance & Trust 0.4 U[0.3, 0.5]


0.15 U[0.11, 0.19]

D3.4 Quality of Governance 0.05 U[0.04, 0.06]

1. Cultural Vibrancy sub-index 0.4 U[0.3, 0.5]

2. Creative Economy sub-index 0.4 U[0.3, 0.5]

3. Enabling Environment sub-index 0.2 U[0.15, 0.25]

The robustness assessment of the C3 Index involved running 2 000 simulations. The simulations explored the issue of weighting and involved 1 000 runs for each normalisation method, each corresponding to a different set of weights for the nine dimensions, randomly sampled from uniform continuous distributions centred in the reference values provided by the experts. A perturbation of the weights ± 25 % around the reference values was applied. The limit values of uncertainty intervals for the dimension weights are shown in Table 3. In all simulations, sampled weights are rescaled so that they always sum to 1.

The effect of normalising all indicators using percentile ranks was also tested because the use of percentile ranks would mean outliers would not to have to be treated.


The main results of the robustness analysis are shown in Figure 1, with median ranks and 90 % confidence intervals computed across the 2 000 Monte Carlo simulations for the C3 Index in the four city size groups11: cities are categorised into four groups based on their population size and ordered within each group according to their reference rank (black line), the dot being the median rank. Yellow bars represent, for each city, the 90 % confidence interval across all simulations.

All the city ranks lie within the simulated intervals, and these are narrow enough for most cities (fewer than ± 3 positions) to allow for meaningful inferences to be drawn about a city’s positioning in the peer-group classification. If the median rank across the simulated scenarios can be considered representative of these 2 000 scenarios, then the fact that the rank is close to the median rank (fewer than two positions away) for 75 % of the cities suggests that the C3 Index is a suitable summary measure for cities’ performance within a peer group. Furthermore, the reasonably narrow confidence intervals for the majority of the cities’ ranks (fewer than ± 3 positions for 85 % of the cities) imply that the ranks are also, for most cities, robust to changes in the dimension weights and the normalisation formula.

The results for the three sub-indices – ‘Cultural Vibrancy’, ‘Creative Economy’, and ‘Enabling Environment’ – are also robust and representative of the plurality of scenarios considered. The ‘Cultural Vibrancy’ rank is close to the median rank (fewer than two positions away) for 75 % of the cities and the rank intervals are ± 3 positions away for 82 % of the cities. Similarly, the ‘Creative Economy’ rank is close to the median rank (fewer than two positions away) for 73 % of the cities, and the rank intervals are ± 3 positions for 82 % of the cities. Finally, the ‘Enabling Environment’ rank is close to the median rank (fewer than two positions away) for 66 % of the cities, and the rank intervals are ± 3 positions for 78 % of the cities.

Overall, city ranks in the C3 Index and its three sub-indices are fairly robust to changes in the dimension weights and the normalisation method, for the majority of the cities analysed. For full transparency and information, Table 4 reports the C3 city ranks (and those of the sub-indices) together with the simulated intervals (90 % of the 2 000 scenarios) for a full appreciation of the robustness of these ranks to the computation methodology.

Figure 1. Robustness analysis (C3 Index rank vs. median rank, 90 % confidence intervals, four city size groups)


NB: The Spearman rank correlation between the median rank and the C3 Index 2019 rank is 0.998. Median ranks and intervals are calculated over 2 000 simulated scenarios combining perturbed weights (± 25 % around the nominal weights assigned by experts), and percentile ranks versus min-max normalisation at the indicator level.

Table 4. City ranks and 90 % confidence intervals for C3 Index 2017 and the three sub-indices (for four city size groups)

XXL group C3 Index 1. Cultural Vibrancy

2. Creative Economy


Paris 1 [1, 1] 1 [1, 1] 1 [1, 1] 2 [1, 4]

Munich 2 [2, 2] 6 [6, 8] 2 [2, 3] 3 [1, 4]

London 3 [3, 8] 13 [12, 16] 4 [4, 7] 1 [1, 3]

Milan 4 [3, 5] 4 [3, 4] 5 [3, 6] 12 [10, 14]

Berlin 5 [4, 6] 5 [5, 5] 7 [5, 7] 8 [8, 13]

Vienna 6 [4, 8] 3 [2, 3] 16 [11, 16] 7 [7, 10]

Budapest 7 [5, 15] 14 [9, 14] 3 [2, 13] 16 [15, 17]

Prague 8 [5, 8] 2 [2, 4] 13 [11, 15] 15 [14, 16]

Barcelona 9 [6, 9] 8 [7, 8] 11 [7, 12] 5 [3, 6]

Hamburg 10 [9, 10] 11 [11, 17] 9 [3, 9] 14 [11, 14]

Madrid 11 [11, 13] 15 [12, 15] 12 [9, 13] 6 [5, 7]

Warsaw 12 [11, 16] 17 [12, 17] 6 [5, 15] 19 [18, 19]

Cologne 13 [10, 14] 10 [10, 11] 14 [9, 14] 11 [8, 13]

Rome 14 [12, 15] 7 [6, 7] 15 [15, 18] 18 [17, 19]

Lyon 15 [11, 15] 9 [9, 10] 18 [17, 18] 13 [8, 13]

Brussels 16 [13, 16] 16 [13, 16] 17 [15, 17] 9 [5, 10]

Bucharest 17 [17, 19] 19 [19, 19] 10 [10, 16] 17 [16, 19]

Rotterdam 18 [17, 18] 12 [10, 17] 19 [19, 19] 10 [9, 15]

Birmingham 19 [18, 20] 18 [18, 18] 20 [20, 20] 4 [3, 5]

Sofia 20 [19, 20] 20 [20, 20] 8 [7, 9] 20 [20, 20]


XL group C3 Index 1. Cultural Vibrancy

2. Creative Economy


Copenhagen 1 [1, 2] 3 [1, 3] 4 [4, 5] 3 [1, 4]

Lisbon 2 [1, 7] 1 [1, 4] 3 [1, 10] 23 [17, 24]

Stockholm 3 [2, 4] 6 [5, 8] 2 [1, 3] 7 [7, 15]

Dublin 4 [2, 5] 2 [2, 3] 22 [12, 22] 1 [1, 5]

Stuttgart 5 [4, 6] 12 [8, 13] 1 [1, 3] 12 [9, 12]

Amsterdam 6 [3, 6] 4 [3, 5] 8 [6, 10] 6 [6, 7]

Glasgow 7 [7, 9] 8 [8, 13] 19 [10, 20] 4 [2, 4]

Dresden 8 [7, 11] 7 [5, 7] 7 [6, 16] 18 [17, 21]

Frankfurt 9 [6, 10] 14 [13, 18] 6 [3, 7] 9 [6, 10]

Helsinki 10 [7, 10] 9 [6, 9] 9 [7, 18] 11 [10, 12]

Manchester 11 [11, 14] 22 [22, 27] 13 [9, 17] 5 [3, 5]

Edinburgh 12 [11, 15] 11 [9, 18] 25 [19, 26] 2 [1, 4]

Nuremberg 13 [10, 14] 18 [11, 18] 11 [9, 20] 13 [11, 13]

Vilnius 14 [12, 20] 28 [27, 28] 5 [4, 7] 25 [25, 32]

Kraków 15 [12, 20] 10 [8, 11] 20 [15, 24] 24 [16, 28]

Gothenburg 16 [14, 19] 23 [21, 27] 10 [5, 12] 15 [14, 26]

Leipzig 17 [16, 20] 16 [15, 18] 16 [14, 19] 29 [28, 31]

Leeds 18 [17, 23] 35 [33, 35] 14 [10, 16] 8 [8, 9]

Toulouse 19 [13, 21] 13 [11, 15] 23 [19, 24] 10 [7, 14]

Bremen 20 [16, 21] 25 [22, 25] 15 [11, 17] 17 [13, 17]

Athens 21 [16, 25] 5 [4, 13] 29 [29, 33] 33 [22, 34]

Poznań 22 [20, 27] 27 [26, 32] 12 [9, 22] 27 [25, 33]

Hannover 23 [17, 24] 24 [23, 26] 21 [11, 22] 20 [13, 22]

Antwerp 24 [20, 25] 15 [12, 17] 24 [24, 28] 21 [16, 23]

Wrocław 25 [23, 29] 31 [29, 32] 17 [12, 26] 35 [32, 35]

The Hague 26 [24, 27] 29 [26, 32] 26 [21, 27] 16 [14, 29]

Riga 27 [26, 34] 33 [28, 34] 18 [10, 32] 40 [39, 40]

Turin 28 [24, 29] 20 [20, 21] 27 [23, 28] 37 [33, 37]

Bordeaux 29 [24, 29] 19 [19, 19] 30 [28, 31] 28 [26, 31]

Valencia 30 [28, 31] 30 [24, 30] 32 [31, 33] 19 [17, 21]

Zagreb 31 [30, 34] 39 [38, 40] 28 [25, 28] 22 [20, 25]

Genoa 32 [30, 34] 17 [14, 18] 35 [33, 38] 39 [39, 40]

Bradford 33 [31, 38] 32 [31, 39] 37 [33, 38] 14 [13, 29]

Essen 34 [30, 35] 36 [35, 36] 31 [28, 31] 26 [21, 29]

Seville 35 [32, 35] 21 [20, 23] 38 [36, 38] 32 [30, 37]

Lille 36 [35, 37] 34 [33, 35] 36 [34, 38] 34 [25, 35]

Marseille 37 [35, 39] 40 [34, 40] 34 [34, 35] 31 [19, 32]

Naples 38 [37, 40] 26 [22, 31] 40 [40, 40] 38 [36, 38]

Zaragoza 39 [37, 40] 37 [36, 37] 39 [39, 39] 30 [27, 38]

Łódź 40 [37, 40] 38 [38, 40] 33 [32, 37] 36 [34, 38]


L group C3 Index 1. Cultural Vibrancy

2. Creative Economy


Florence 1 [1, 9] 1 [1, 2] 17 [15, 18] 34 [25, 35]

Karlsruhe 2 [1, 4] 14 [11, 16] 4 [1, 4] 3 [3, 5]

Venice 3 [2, 18] 2 [1, 2] 26 [25, 31] 32 [29, 33]

Bristol 4 [1, 5] 11 [9, 12] 6 [3, 6] 5 [3, 5]

Tallinn 5 [1, 6] 6 [4, 6] 3 [1, 5] 26 [20, 27]

Brighton 6 [4, 8] 10 [10, 14] 7 [7, 8] 2 [1, 3]

Eindhoven 7 [3, 10] 17 [15, 18] 5 [3, 8] 7 [7, 9]

Graz 8 [5, 10] 4 [3, 5] 12 [11, 15] 9 [8, 12]

Bratislava 9 [6, 14] 12 [8, 16] 1 [1, 8] 33 [26, 34]

Malmö 10 [8, 18] 18 [18, 21] 2 [1, 4] 21 [17, 29]

Utrecht 11 [4, 12] 15 [13, 15] 9 [5, 10] 4 [2, 5]

Bologna 12 [9, 14] 3 [3, 6] 13 [11, 15] 25 [19, 29]

Ghent 13 [7, 15] 5 [3, 5] 20 [17, 22] 19 [12, 20]

Nottingham 14 [10, 16] 19 [18, 21] 15 [11, 16] 1 [1, 2]

Mannheim 15 [11, 17] 16 [16, 17] 10 [8, 10] 14 [11, 16]

Ljubljana 16 [14, 17] 13 [13, 15] 11 [9, 13] 13 [12, 16]

Nantes 17 [14, 18] 7 [7, 10] 19 [13, 20] 16 [15, 19]

Montpellier 18 [11, 19] 8 [8, 11] 21 [10, 21] 12 [9, 15]

Aarhus 19 [17, 19] 9 [7, 9] 24 [21, 28] 11 [9, 18]

Liverpool 20 [20, 21] 26 [26, 29] 22 [19, 23] 6 [6, 6]

Espoo 21 [21, 25] 32 [31, 32] 18 [17, 21] 8 [7, 9]

Gdańsk 22 [21, 25] 28 [28, 30] 14 [11, 22] 22 [19, 24]

Sintra 23 [20, 33] 31 [29, 37] 8 [8, 27] 37 [35, 38]

Kaunas 24 [21, 26] 24 [24, 28] 16 [14, 19] 29 [27, 33]

Cluj-Napoca 25 [25, 32] 27 [25, 29] 23 [21, 33] 20 [16, 35]

Leiden 26 [21, 27] 22 [22, 23] 35 [26, 36] 10 [9, 11]

Brno 27 [22, 28] 21 [19, 21] 30 [28, 33] 18 [17, 24]

Bilbao 28 [27, 29] 23 [20, 25] 28 [25, 29] 27 [24, 31]

Katowice 29 [26, 29] 30 [27, 32] 25 [22, 25] 24 [22, 30]

Liège 30 [24, 31] 25 [22, 26] 34 [30, 35] 23 [14, 23]

Timişoara 31 [29, 32] 35 [34, 36] 27 [24, 28] 17 [16, 22]

Thessaloniki 32 [29, 33] 20 [17, 23] 36 [35, 38] 35 [25, 37]

Bochum 33 [30, 33] 38 [34, 38] 31 [29, 34] 15 [10, 15]

Iaşi 34 [34, 35] 33 [33, 34] 33 [31, 36] 28 [24, 33]

Las Palmas 35 [34, 36] 36 [31, 36] 32 [31, 34] 36 [32, 39]

Cordova 36 [35, 37] 29 [24, 30] 39 [38, 40] 31 [30, 35]

Varna 37 [36, 37] 37 [37, 38] 29 [25, 31] 40 [40, 40]

Lublin 38 [38, 39] 40 [40, 40] 38 [36, 39] 30 [26, 33]

Ostrava 39 [38, 40] 34 [33, 37] 40 [39, 40] 38 [36, 38]

Plovdiv 40 [39, 40] 39 [39, 39] 37 [36, 38] 39 [37, 39]


S-M group C3 Index 1. Cultural Vibrancy

2. Creative Economy


Lund 1 [1, 3] 13 [12, 19] 1 [1, 2] 8 [2, 8]

Weimar 2 [1, 8] 1 [1, 4] 19 [15, 28] 25 [21, 27]

Heidelberg 3 [1, 3] 7 [5, 10] 3 [2, 4] 5 [1, 6]

Cork 4 [4, 6] 3 [2, 3] 44 [36, 45] 2 [2, 10]

Tartu 5 [3, 5] 4 [1, 4] 12 [6, 14] 34 [24, 39]

Avignon 6 [6, 18] 2 [2, 10] 35 [28, 37] 39 [24, 41]

Galway 7 [6, 23] 5 [5, 7] 62 [59, 63] 1 [1, 8]

Mainz 8 [4, 11] 44 [35, 46] 2 [1, 3] 9 [7, 10]

Porto 9 [7, 15] 9 [8, 25] 8 [6, 10] 50 [33, 51]

Leuven 10 [7, 14] 42 [37, 44] 4 [4, 15] 7 [2, 9]

York 11 [8, 12] 15 [14, 32] 26 [13, 28] 6 [4, 7]

Groningen 12 [5, 15] 17 [11, 23] 16 [7, 18] 21 [12, 21]

Trento 13 [9, 17] 6 [2, 6] 41 [34, 47] 42 [27, 48]

Turku 14 [12, 16] 16 [15, 24] 30 [21, 36] 11 [7, 13]

Linz 15 [7, 17] 33 [17, 35] 6 [5, 12] 18 [16, 22]

Norwich 16 [11, 17] 23 [22, 26] 32 [21, 39] 3 [2, 6]

Limerick 17 [16, 25] 19 [10, 19] 57 [51, 58] 4 [3, 11]

Granada 18 [17, 30] 10 [9, 13] 51 [49, 54] 29 [19, 29]

Amersfoort 19 [16, 30] 50 [44, 55] 7 [4, 9] 15 [13, 56]

Kortrijk 20 [18, 30] 26 [20, 38] 17 [12, 22] 28 [24, 39]

Tampere 21 [17, 27] 55 [40, 56] 9 [6, 25] 14 [12, 18]

Pula 22 [19, 53] 12 [12, 32] 39 [30, 64] 46 [40, 66]

‘s-Hertogenbosch 23 [11, 26] 48 [43, 49] 10 [4, 11] 19 [18, 25]

Santiago 24 [18, 26] 18 [8, 18] 33 [23, 38] 38 [37, 43]

Faro 25 [19, 36] 31 [28, 35] 5 [3, 21] 70 [68, 71]

Trieste 26 [21, 38] 8 [7, 12] 52 [46, 52] 68 [57, 68]

Maastricht 27 [15, 29] 25 [15, 26] 50 [26, 50] 17 [13, 17]

Uppsala 28 [24, 38] 65 [65, 70] 11 [7, 17] 16 [14, 18]

Coimbra 29 [16, 31] 22 [8, 26] 20 [17, 24] 53 [44, 55]

Waterford 30 [26, 38] 27 [24, 33] 54 [50, 55] 13 [12, 18]

Odense 31 [21, 32] 45 [33, 49] 40 [34, 43] 10 [6, 11]

Bruges 32 [27, 40] 11 [7, 13] 47 [33, 49] 72 [71, 76]

Umeå 33 [29, 40] 56 [51, 58] 13 [8, 15] 37 [34, 54]

Leeuwarden 34 [25, 36] 40 [33, 40] 48 [30, 49] 20 [19, 29]

Norrköping 35 [30, 41] 62 [60, 68] 14 [9, 18] 30 [27, 36]

Dundee 36 [27, 41] 43 [27, 44] 56 [49, 57] 12 [11, 15]

Perugia 37 [29, 41] 30 [17, 32] 23 [20, 43] 69 [58, 70]

Cagliari 38 [30, 42] 20 [12, 22] 37 [30, 53] 65 [50, 65]

San Sebastián-Donostia

39 [29, 43] 36 [24, 38] 38 [26, 42] 44 [41, 54]

Klaipėda 40 [36, 51] 52 [46, 57] 15 [15, 37] 49 [45, 67]


S-M group C3 Index 1. Cultural Vibrancy

2. Creative Economy


Parma 41 [24, 44] 37 [23, 38] 25 [19, 33] 62 [53, 63]

Mons 42 [38, 46] 38 [34, 55] 34 [24, 41] 51 [32, 54]

Pesaro 43 [38, 49] 24 [21, 30] 36 [31, 53] 73 [71, 75]

Limoges 44 [37, 59] 58 [54, 58] 28 [22, 60] 27 [24, 37]

Saint-Étienne 45 [38, 52] 21 [19, 45] 63 [59, 64] 47 [29, 48]

Salamanca 46 [39, 52] 28 [21, 30] 73 [72, 77] 23 [16, 25]

Szeged 47 [37, 51] 64 [56, 64] 24 [16, 29] 33 [30, 37]

Brescia 48 [38, 52] 49 [48, 51] 22 [13, 28] 67 [58, 68]

Namur 49 [44, 53] 47 [46, 66] 49 [39, 50] 40 [21, 43]

Győr 50 [45, 59] 61 [58, 68] 18 [16, 44] 60 [57, 71]

Maribor 51 [41, 55] 67 [59, 67] 45 [30, 46] 24 [19, 26]

Matera 52 [46, 69] 14 [11, 22] 69 [68, 73] 79 [79, 79]

Lecce 53 [46, 56] 34 [32, 41] 53 [49, 58] 63 [55, 65]

Veszprém 54 [44, 58] 60 [52, 62] 29 [19, 40] 54 [52, 59]

Osijek 55 [50, 74] 35 [28, 71] 77 [75, 77] 31 [29, 42]

Nitra 56 [54, 60] 46 [45, 49] 59 [55, 64] 45 [38, 52]

Ostend 57 [53, 61] 39 [36, 44] 67 [65, 67] 43 [38, 46]

Guimarães 58 [49, 62] 69 [67, 70] 21 [16, 46] 61 [59, 66]

Ravenna 59 [53, 66] 32 [25, 45] 55 [53, 61] 77 [75, 77]

Sibiu 60 [56, 63] 54 [53, 66] 64 [56, 64] 32 [30, 40]

Liepāja 61 [53, 65] 57 [42, 62] 43 [29, 62] 59 [57, 74]

Pécs 62 [47, 64] 70 [59, 70] 27 [21, 42] 58 [58, 65]

Split 63 [59, 69] 41 [33, 42] 78 [78, 78] 36 [33, 50]

Lleida 64 [56, 65] 73 [70, 74] 61 [56, 61] 22 [20, 29]

Terrassa 65 [58, 70] 76 [75, 78] 46 [25, 50] 35 [31, 41]

Karlovy Vary 66 [64, 75] 29 [28, 49] 75 [74, 77] 78 [78, 78]

Veliko Tarnovo 67 [66, 71] 51 [50, 52] 66 [65, 68] 64 [57, 69]

Rijeka 68 [66, 72] 68 [61, 68] 74 [74, 76] 26 [26, 38]

Burgos 69 [67, 73] 59 [55, 69] 65 [65, 67] 56 [50, 71]

Olomouc 70 [60, 73] 53 [43, 60] 71 [67, 72] 57 [45, 61]

Braga 71 [54, 72] 77 [75, 78] 31 [20, 40] 55 [48, 56]

Pilsen 72 [64, 74] 63 [55, 63] 60 [56, 64] 71 [60, 73]

Debrecen 73 [63, 74] 79 [79, 79] 42 [23, 46] 48 [41, 48]

Toruń 74 [62, 75] 75 [72, 76] 58 [44, 64] 52 [47, 57]

Baia Mare 75 [74, 76] 74 [73, 76] 70 [70, 72] 41 [38, 54]

Košice 76 [74, 76] 71 [71, 73] 72 [68, 73] 66 [62, 67]

Prešov 77 [77, 77] 72 [72, 74] 68 [68, 71] 74 [72, 75]

Kalamata 78 [78, 78] 66 [60, 66] 79 [79, 79] 76 [74, 76]

Patras 79 [79, 79] 78 [77, 78] 76 [72, 77] 75 [68, 77]


Conclusions

Overall, the analysis of statistical coherence reveals that the statistical structure of the C3 Index 2019 is coherent with its conceptual framework, given that all indicators have good-to-strong correlations with their respective dimensions. Furthermore, all dimensions correlate strongly with the three sub-indices and the C3 Index itself and are fairly in line with the expert-based weights, all of which indicates that the framework is well balanced.

The C3 Index and all three sub-indices are relatively robust to methodological assumptions relating to the normalisation method and the dimension weights. It is reassuring that for about 80 % of the cities included in the C3 Index, the overall and sub-index ranks are the result of the underlying data and not of the modelling choices. Consequently, inferences can be drawn for most cities within their peer group, particularly in the XXL group where the 90 % confidence interval is in most cases very small (2 positions), with the exception of Budapest, which may go from the 15th up to the fifth position, based on the calculated 90 % confidence interval. More caution may be needed for a few cities, such as Vilnius, Kraków, Toulouse, Athens and Riga in the XL group (with 90 % confidence interval widths of 8-9 positions), Malmö, Sintra and Venice in the L group (with 90 % confidence interval widths of 10, 13 and 16 positions, respectively) and 13 cities in the S-M group – Coimbra, Klaipėda, ‘s-Hertogenbosch, Faro, Trieste, Galway, Pécs, Braga, Parma, Limoges, Matera, Osijek and Pula – with 90 % confidence interval widths between 15 and 24 positions, and Pula in particular is subject to an even more uncertain result, with a 90 % confidence interval of 34 positions.


Appendix

Table A1. Cultural and Creative Cities Monitor indicators

Variable name

Short explanation Geo level

Reference period

Mode year

Availability Source

Sub-index 1 Cultural Vibrancy

Dimension 1.1 Cultural Venues & Facilities

1. Sights & landmarks

Points of historical, cultural and/or artistic interest, such as architectural buildings, religious sites, monuments and statues, churches and cathedrals, bridges, towers and fountains, amongst other things, divided by the total population and then multiplied by 100 000.

City 2019 2019 100 % TripAdvisor

2. Museums & art galleries

Number of museums that are open to the public divided by the total population and then multiplied by 100 000.


3. Cinemas Number of cinemas in the city divided by the total population and then multiplied by 100 000.

City 2019 2019 99 % OpenStreetMap

4. Concert & music halls

Number of theatres and other music venues (concert halls, clubs, etc.) and current shows divided by the total population and then multiplied by 100 000.


5.Theatres Number of theatres in the city divided by the total population and then multiplied by 100 000.

City 2019 2019 99 % OpenStreetMap

Dimension 1.2 Cultural Participation & Attractiveness

6. Tourist overnight stays

Total annual number of nights that tourists/guests have spent in tourist accommodation establishments (hotel or similar) divided by the total population.

City 2011-2017 2017 87 % Eurostat (Urban Audit)

7. Museum visitors Total number of museum tickets sold during the reference year divided by the total population and then multiplied by 1 000.


8. Cinema attendance

Total number of tickets sold, referring to all films screened during the year, divided by the total population and then multiplied by 1 000.


9. Satisfaction with cultural facilities

Percentage of population that is very satisfied with cultural facilities in the city.

City 2015 2015 32 % Flash Eurobarometer

366 by TNS/European

Commission (Survey on

‘Quality of life in cities’)


Variable name


Reference period

Mode year

Availability Source

Sub-index 2 Creative Economy

Dimension 2.1 Creative & Knowledge-based Jobs

10. Jobs in arts, culture & entertainment

Number of jobs in arts-, culture- and entertainment-related activities such as performing arts, museums and libraries, divided by the total population and then multiplied by 1 000 (NACE Rev. 2, R-U).


11. Jobs in media & communication

Number of jobs in media- and communication-related activities such as book and music publishing, film production and TV, divided by the total population and then multiplied by 1 000 (NACE Rev. 2, J).


12. Jobs in other creative sectors

Number of jobs in professional, scientific and technical, administrative and support service activities such as architecture, advertising, design and photographic activities, divided by the total population and then multiplied by 1 000 (NACE Rev. 2, M-N).


Dimension 2.2 Intellectual Property & Innovation

13. ICT patent applications

Three-year average number of ICT patent applications (including: consumer electronics, computers and office machinery, and telecommunications) filed to the European Patent Office by priority year, divided by the total population and then multiplied by 1 million.

NUTS 3 2013-2015 2013-2015

94 % OECD REGPAT

14. Community design applications

Three-year average number of Community design applications filed to the European Union Intellectual Property Office, divided by the total population and then multiplied by 1 million.

NUTS 3 2014-2016 2014-2016

97 % Eurostat (Regional Statistics)

Dimension 2.3 New Jobs in Creative Sectors

15. Jobs in new arts, culture & entertainment enterprises

Number of persons employed in the enterprises established in the reference year in arts, culture and entertainment activities such as performing arts, museums and libraries, divided by the total population and then multiplied by 100 000.

NUTS 3 2010-2016 2016 63 % Eurostat (Regional Statistics)

16. Jobs in new media & communication enterprises

Number of persons employed in the enterprises established in the reference year in in media and communication activities such as book and music publishing, film production and TV, divided by the total population and then multiplied by 100 000.


17. Jobs in new enterprises in other creative sectors

Number of persons employed in the enterprises established in the reference year in professional, scientific and technical activities such as architecture, advertising, design and photographic activities, divided by the total population and then multiplied by 100 000.



Variable name


Reference period

Mode year

Availability Source

Sub-index 3 Enabling Environment

Dimension 3.1 Human Capital & Education

18. Graduates in arts & humanities

Number of tertiary education graduates (ISCED 2011 levels 5-8) in arts and humanities courses, divided by the total population and then multiplied by 100 000.

City 2013-2015 2015 89 % European Tertiary

Education Register (ETER)

project

19. Graduates in ICT

Number of tertiary education graduates (ISCED 2011 levels 5-8) in Information and Communication Technologies courses, divided by the total population and then multiplied by 100 000.

City 2013-2015 2015 89 % ETER project

20. Average appearances in university rankings

Average number of universities’ appearances in four different university rankings: QS, Shanghai, Leiden and Times.

City 2018 2018 100 % QS, Shanghai, Leiden, Times

rankings

Dimension 3.2 Openness, Tolerance & Trust

21. Foreign graduates

Percentage of the total number of tertiary education graduates (ISCED 2011 levels 5-8) who are foreigners.

City 2013-2015 2015 89 % ETER project

22. Foreign-born population

Percentage of the total population that is foreign-born.


23. Tolerance of foreigners

Percentage of the population that very strongly agrees with the statement: ‘The presence of foreigners is good for this city’.


366 by TNS/European



24. Integration of foreigners

Percentage of the population that very strongly agrees with the statement: ‘Foreigners who live in this city are well integrated’.


366 by TNS/European



25. People trust Percentage of the population that very strongly agrees with the statement: ‘Generally speaking, most people in this city can be trusted’.


366 by TNS/European



Dimension 3.3 Local & International Connections

26. Accessibility to flights

Population-weighted average number of accessible passenger flights per day, within 1 hour 30 minutes of travel by road.

City 2016 2016 99 % Directorate-General for

Regional and Urban Policy

27. Accessibility by road

Population accessible within 1 hour 30 minutes by road, as share of the population in a 120 km radius.



28. Accessibility by train

Population accessible within 1 hour 30 minutes by rail (average total travel time), as share of the population in a 120 km radius.




Variable name


Reference period

Mode year

Availability Source

Dimension 3.4 Quality of Governance

29. Quality of governance

Computed indicator measuring the quality of government in three areas of public services: education, healthcare and law enforcement.

NUTS 2, NUTS 1

and NUTS 0

2017 2017 96 % Directorate-General for


Notes

A Community design is a unitary industrial design right that covers the European Union. A design is defined as the appearance of the whole or a part of a product resulting from the features of, in particular, the lines, contours, colours, shape, texture and/or materials of the product itself and/or its ornamentation.

ISCED 5: short-cycle tertiary education. ISCED 6: Bachelor’s degree or programmes of an equivalent level. ISCED 7: Master’s degree or programme of an equivalent level. ISCED 8: Doctorate or programmes of an equivalent level.

NACE: the statistical classification of economic activities in the European Union. NACE is a four-digit classification providing the framework for collecting and presenting a large range of statistical data according to economic activity in the fields of economic statistics (e.g. production, employment and national accounts) and in other statistical domains developed within the European statistical system (ESS). NACE Rev. 2, a revised classification, was adopted at the end of 2006 and applied from 2007 onwards.

http://ec.europa.eu/eurostat/statistics-explained/index.php/Glossary:Statistical_classification_of_economic_activities_in_the_European_Community_(NACE)




Table A2. Data coverage

DATA COVERAGE DATA COVERAGE DATA COVERAGE DATA COVERAGE

City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


Vienna 83 % 67 % 63 % 100 % Aarhus 79 % 78 % 100 % 58 % Veszprém 59 % 67 % 63 % 42 % Wrocław 86 % 78 % 100 % 75 %

Graz 83 % 67 % 63 % 100 % Odense 83 % 78 % 100 % 67 % Dublin 86 % 78 % 63 % 100 % Poznań 86 % 78 % 100 % 75 %

Linz 69 % 56 % 63 % 75 % Tallinn 100 % 89 % 100 % 100 % Cork 72 % 67 % 63 % 75 % Gdańsk 100 % 89 % 100 % 100 %

Brussels 79 % 89 % 63 % 75 % Tartu 86 % 78 % 100 % 75 % Limerick 66 % 44 % 63 % 75 % Lublin 86 % 78 % 100 % 75 %

Antwerp 90 % 89 % 63 % 100 % Athens 83 % 67 % 63 % 100 % Galway 69 % 56 % 63 % 75 % Katowice 86 % 78 % 100 % 75 %

Ghent 76 % 78 % 63 % 75 % Thessaloniki 69 % 56 % 63 % 75 % Waterford 69 % 56 % 63 % 75 % Toruń 86 % 78 % 100 % 75 %

Liège 79 % 89 % 63 % 75 % Patras 66 % 56 % 50 % 75 % Rome 93 % 78 % 100 % 92 % Lisbon 100 % 89 % 100 % 100 %

Bruges 76 % 78 % 63 % 75 % Kalamata 59 % 44 % 50 % 67 % Milan 79 % 67 % 100 % 67 % Porto 86 % 78 % 100 % 75 %

Namur 66 % 78 % 63 % 50 % Madrid 100 % 89 % 100 % 100 % Naples 93 % 78 % 100 % 92 % Braga 100 % 89 % 100 % 100 %

Leuven 76 % 78 % 63 % 75 % Barcelona 100 % 89 % 100 % 100 % Turin 93 % 78 % 100 % 92 % Coimbra 83 % 78 % 88 % 75 %

Mons 66 % 78 % 63 % 50 % Valencia 86 % 78 % 100 % 75 % Genoa 79 % 67 % 100 % 67 % Faro 86 % 78 % 100 % 75 %

Kortrijk 76 % 78 % 63 % 75 % Seville 86 % 78 % 100 % 75 % Florence 79 % 67 % 100 % 67 % Sintra 86 % 78 % 100 % 75 %

Ostend 69 % 78 % 63 % 58 % Zaragoza 86 % 78 % 100 % 75 % Bologna 93 % 78 % 100 % 92 % Guimarães 76 % 78 % 100 % 50 %

Sofia 100 % 89 % 100 % 100 % Las Palmas 86 % 78 % 100 % 75 % Venice 79 % 67 % 100 % 67 % Bucharest 72 % 78 % 63 % 67 %

Plovdiv 86 % 78 % 100 % 75 % Santiago 76 % 78 % 100 % 50 % Trento 79 % 67 % 100 % 67 % Cluj-Napoca 72 % 78 % 63 % 67 %

Geneva 86 % 89 % 63 % 92 % Bilbao 76 % 78 % 100 % 50 % Trieste 79 % 67 % 100 % 67 % Timișoara 59 % 67 % 63 % 42 %

Bern 72 % 78 % 63 % 67 % Cordova 86 % 78 % 100 % 75 % Perugia 79 % 67 % 100 % 67 % Sibiu 59 % 67 % 63 % 42 %

Basel 72 % 78 % 63 % 67 % Granada 86 % 78 % 100 % 75 % Cagliari 79 % 67 % 100 % 67 % Iași 59 % 67 % 63 % 42 %

Zurich 86 % 89 % 63 % 92 % San Sebastián-Donostia 76 % 78 % 100 % 50 % Brescia 79 % 67 % 100 % 67 % Baia Mare 55 % 67 % 50 % 42 %

Nicosia 69 % 56 % 25 % 100 % Terrassa 76 % 78 % 100 % 50 % Lecce 79 % 67 % 100 % 67 % Stockholm 90 % 89 % 63 % 100 %

Limassol 55 % 44 % 25 % 75 % Burgos 83 % 78 % 88 % 75 % Pesaro 79 % 67 % 100 % 67 % Gothenburg 76 % 78 % 63 % 75 %

Varna 83 % 78 % 88 % 75 % Salamanca 86 % 78 % 100 % 75 % Matera 76 % 56 % 100 % 67 % Malmö 90 % 89 % 63 % 100 %

Veliko Tarnovo 83 % 78 % 88 % 75 % Lleida 83 % 67 % 100 % 75 % Parma 79 % 67 % 100 % 67 % Umeå 76 % 78 % 63 % 75 %

Prague 90 % 89 % 63 % 100 % Helsinki 100 % 89 % 100 % 100 % Ravenna 79 % 67 % 100 % 67 % Uppsala 76 % 78 % 63 % 75 %

Brno 76 % 78 % 63 % 75 % Tampere 86 % 78 % 100 % 75 % Vilnius 100 % 89 % 100 % 100 % Norrköping 69 % 78 % 63 % 58 %

Ostrava 90 % 89 % 63 % 100 % Turku 86 % 78 % 100 % 75 % Luxembourg 59 % 56 % 25 % 75 % Lund 72 % 67 % 63 % 75 %

Pilsen 76 % 78 % 63 % 75 % Espoo 86 % 78 % 100 % 75 % Kaunas 86 % 78 % 100 % 75 % Ljubljana 90 % 89 % 63 % 100 %

Olomouc 76 % 78 % 63 % 75 % Paris 100 % 89 % 100 % 100 % Klaipeda 83 % 78 % 88 % 75 % Maribor 76 % 78 % 63 % 75 %

Karlovy Vary 76 % 78 % 63 % 75 % Lyon 86 % 78 % 100 % 75 % Riga 100 % 89 % 100 % 100 % Bratislava 93 % 89 % 75 % 100 %

Berlin 90 % 89 % 63 % 100 % Toulouse 86 % 78 % 100 % 75 % Valletta 59 % 67 % 25 % 67 % Košice 93 % 89 % 75 % 100 %

Hamburg 90 % 89 % 63 % 100 % Bordeaux 100 % 89 % 100 % 100 % Liepāja 100 % 89 % 100 % 100 % Nitra 86 % 78 % 100 % 75 %

Munich 90 % 89 % 63 % 100 % Nantes 86 % 78 % 100 % 75 % The Hague 76 % 56 % 100 % 67 % Prešov 86 % 78 % 100 % 75 %

Cologne 76 % 78 % 63 % 75 % Lille 93 % 89 % 100 % 83 % Amsterdam 79 % 56 % 100 % 75 % London (Greater) 79 % 56 % 63 % 100 %

Frankfurt 90 % 89 % 63 % 100 % Montpellier 86 % 78 % 100 % 75 % Rotterdam 79 % 56 % 100 % 75 % Birmingham 69 % 56 % 63 % 75 %

Essen 90 % 89 % 63 % 100 % Saint-Etienne 86 % 78 % 100 % 75 % Utrecht 79 % 56 % 100 % 75 % Leeds 66 % 44 % 63 % 75 %

Stuttgart 90 % 89 % 63 % 100 % Limoges 86 % 78 % 100 % 75 % Eindhoven 76 % 44 % 100 % 75 % Glasgow 79 % 56 % 63 % 100 %

Leipzig 90 % 89 % 63 % 100 % Avignon 86 % 78 % 100 % 75 % Groningen 90 % 56 % 100 % 100 % Bradford 66 % 44 % 63 % 75 %

Dresden 86 % 89 % 63 % 92 % Marseille 100 % 89 % 100 % 100 % Leeuwarden 72 % 44 % 88 % 75 % Liverpool 66 % 44 % 63 % 75 %

Bremen 72 % 67 % 63 % 75 % Zagreb 97 % 89 % 88 % 100 % 's-Hertogenbosch 66 % 44 % 100 % 50 % Edinburgh 66 % 44 % 63 % 75 %

Hannover 76 % 78 % 63 % 75 % Rijeka 86 % 78 % 100 % 75 % Amersfoort 76 % 44 % 100 % 75 % Manchester 86 % 78 % 63 % 100 %

Nuremberg 86 % 89 % 63 % 92 % Osijek 79 % 78 % 75 % 75 % Maastricht 79 % 56 % 100 % 75 % Bristol 69 % 56 % 63 % 75 %

Bochum 76 % 78 % 63 % 75 % Split 83 % 78 % 88 % 75 % Oslo 69 % 67 % 25 % 92 % Nottingham 72 % 67 % 63 % 75 %

Weimar 69 % 67 % 50 % 75 % Pula 69 % 78 % 75 % 50 % Stavanger 55 % 56 % 25 % 67 % Brighton and Hove 66 % 44 % 63 % 75 %

Karlsruhe 86 % 89 % 63 % 92 % Budapest 100 % 89 % 100 % 100 % Bergen 55 % 56 % 25 % 67 % York 69 % 56 % 63 % 75 %

Mainz 72 % 67 % 63 % 75 % Pécs 86 % 78 % 100 % 75 % Leiden 76 % 44 % 100 % 75 % Dundee 66 % 44 % 63 % 75 %

Mannheim 83 % 78 % 63 % 92 % Debrecen 86 % 78 % 100 % 75 % Warsaw 100 % 89 % 100 % 100 % Norwich 69 % 56 % 63 % 75 %

Heidelberg 72 % 67 % 63 % 75 % Szeged 86 % 78 % 100 % 75 % Łódź 86 % 78 % 100 % 75 %

Copenhagen 97 % 89 % 100 % 92 % Győr 83 % 67 % 100 % 75 % Kraków 100 % 89 % 100 % 100 %

Note: cities in red have not been ranked due to poor data coverage (<33 % on 'Cultural Vibrancy' or 'Creative Economy'); cities in light blue have not been ranked because outside EU countries.


DATA COVERAGE DATA COVERAGE DATA COVERAGE DATA COVERAGE

City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


City C3 Index


2. Creative Economy


Vienna 83 % 67 % 63 % 100 % Aarhus 79 % 78 % 100 % 58 % Veszprém 59 % 67 % 63 % 42 % Wrocław 86 % 78 % 100 % 75 %

Graz 83 % 67 % 63 % 100 % Odense 83 % 78 % 100 % 67 % Dublin 86 % 78 % 63 % 100 % Poznań 86 % 78 % 100 % 75 %

Linz 69 % 56 % 63 % 75 % Tallinn 100 % 89 % 100 % 100 % Cork 72 % 67 % 63 % 75 % Gdańsk 100 % 89 % 100 % 100 %

Brussels 79 % 89 % 63 % 75 % Tartu 86 % 78 % 100 % 75 % Limerick 66 % 44 % 63 % 75 % Lublin 86 % 78 % 100 % 75 %

Antwerp 90 % 89 % 63 % 100 % Athens 83 % 67 % 63 % 100 % Galway 69 % 56 % 63 % 75 % Katowice 86 % 78 % 100 % 75 %

Ghent 76 % 78 % 63 % 75 % Thessaloniki 69 % 56 % 63 % 75 % Waterford 69 % 56 % 63 % 75 % Toruń 86 % 78 % 100 % 75 %

Liège 79 % 89 % 63 % 75 % Patras 66 % 56 % 50 % 75 % Rome 93 % 78 % 100 % 92 % Lisbon 100 % 89 % 100 % 100 %

Bruges 76 % 78 % 63 % 75 % Kalamata 59 % 44 % 50 % 67 % Milan 79 % 67 % 100 % 67 % Porto 86 % 78 % 100 % 75 %

Namur 66 % 78 % 63 % 50 % Madrid 100 % 89 % 100 % 100 % Naples 93 % 78 % 100 % 92 % Braga 100 % 89 % 100 % 100 %

Leuven 76 % 78 % 63 % 75 % Barcelona 100 % 89 % 100 % 100 % Turin 93 % 78 % 100 % 92 % Coimbra 83 % 78 % 88 % 75 %

Mons 66 % 78 % 63 % 50 % Valencia 86 % 78 % 100 % 75 % Genoa 79 % 67 % 100 % 67 % Faro 86 % 78 % 100 % 75 %

Kortrijk 76 % 78 % 63 % 75 % Seville 86 % 78 % 100 % 75 % Florence 79 % 67 % 100 % 67 % Sintra 86 % 78 % 100 % 75 %

Ostend 69 % 78 % 63 % 58 % Zaragoza 86 % 78 % 100 % 75 % Bologna 93 % 78 % 100 % 92 % Guimarães 76 % 78 % 100 % 50 %

Sofia 100 % 89 % 100 % 100 % Las Palmas 86 % 78 % 100 % 75 % Venice 79 % 67 % 100 % 67 % Bucharest 72 % 78 % 63 % 67 %

Plovdiv 86 % 78 % 100 % 75 % Santiago 76 % 78 % 100 % 50 % Trento 79 % 67 % 100 % 67 % Cluj-Napoca 72 % 78 % 63 % 67 %

Geneva 86 % 89 % 63 % 92 % Bilbao 76 % 78 % 100 % 50 % Trieste 79 % 67 % 100 % 67 % Timișoara 59 % 67 % 63 % 42 %

Bern 72 % 78 % 63 % 67 % Cordova 86 % 78 % 100 % 75 % Perugia 79 % 67 % 100 % 67 % Sibiu 59 % 67 % 63 % 42 %

Basel 72 % 78 % 63 % 67 % Granada 86 % 78 % 100 % 75 % Cagliari 79 % 67 % 100 % 67 % Iași 59 % 67 % 63 % 42 %

Zurich 86 % 89 % 63 % 92 % San Sebastián-Donostia 76 % 78 % 100 % 50 % Brescia 79 % 67 % 100 % 67 % Baia Mare 55 % 67 % 50 % 42 %

Nicosia 69 % 56 % 25 % 100 % Terrassa 76 % 78 % 100 % 50 % Lecce 79 % 67 % 100 % 67 % Stockholm 90 % 89 % 63 % 100 %

Limassol 55 % 44 % 25 % 75 % Burgos 83 % 78 % 88 % 75 % Pesaro 79 % 67 % 100 % 67 % Gothenburg 76 % 78 % 63 % 75 %

Varna 83 % 78 % 88 % 75 % Salamanca 86 % 78 % 100 % 75 % Matera 76 % 56 % 100 % 67 % Malmö 90 % 89 % 63 % 100 %

Veliko Tarnovo 83 % 78 % 88 % 75 % Lleida 83 % 67 % 100 % 75 % Parma 79 % 67 % 100 % 67 % Umeå 76 % 78 % 63 % 75 %

Prague 90 % 89 % 63 % 100 % Helsinki 100 % 89 % 100 % 100 % Ravenna 79 % 67 % 100 % 67 % Uppsala 76 % 78 % 63 % 75 %

Brno 76 % 78 % 63 % 75 % Tampere 86 % 78 % 100 % 75 % Vilnius 100 % 89 % 100 % 100 % Norrköping 69 % 78 % 63 % 58 %

Ostrava 90 % 89 % 63 % 100 % Turku 86 % 78 % 100 % 75 % Luxembourg 59 % 56 % 25 % 75 % Lund 72 % 67 % 63 % 75 %

Pilsen 76 % 78 % 63 % 75 % Espoo 86 % 78 % 100 % 75 % Kaunas 86 % 78 % 100 % 75 % Ljubljana 90 % 89 % 63 % 100 %

Olomouc 76 % 78 % 63 % 75 % Paris 100 % 89 % 100 % 100 % Klaipeda 83 % 78 % 88 % 75 % Maribor 76 % 78 % 63 % 75 %

Karlovy Vary 76 % 78 % 63 % 75 % Lyon 86 % 78 % 100 % 75 % Riga 100 % 89 % 100 % 100 % Bratislava 93 % 89 % 75 % 100 %

Berlin 90 % 89 % 63 % 100 % Toulouse 86 % 78 % 100 % 75 % Valletta 59 % 67 % 25 % 67 % Košice 93 % 89 % 75 % 100 %

Hamburg 90 % 89 % 63 % 100 % Bordeaux 100 % 89 % 100 % 100 % Liepāja 100 % 89 % 100 % 100 % Nitra 86 % 78 % 100 % 75 %

Munich 90 % 89 % 63 % 100 % Nantes 86 % 78 % 100 % 75 % The Hague 76 % 56 % 100 % 67 % Prešov 86 % 78 % 100 % 75 %

Cologne 76 % 78 % 63 % 75 % Lille 93 % 89 % 100 % 83 % Amsterdam 79 % 56 % 100 % 75 % London (Greater) 79 % 56 % 63 % 100 %

Frankfurt 90 % 89 % 63 % 100 % Montpellier 86 % 78 % 100 % 75 % Rotterdam 79 % 56 % 100 % 75 % Birmingham 69 % 56 % 63 % 75 %

Essen 90 % 89 % 63 % 100 % Saint-Etienne 86 % 78 % 100 % 75 % Utrecht 79 % 56 % 100 % 75 % Leeds 66 % 44 % 63 % 75 %

Stuttgart 90 % 89 % 63 % 100 % Limoges 86 % 78 % 100 % 75 % Eindhoven 76 % 44 % 100 % 75 % Glasgow 79 % 56 % 63 % 100 %

Leipzig 90 % 89 % 63 % 100 % Avignon 86 % 78 % 100 % 75 % Groningen 90 % 56 % 100 % 100 % Bradford 66 % 44 % 63 % 75 %

Dresden 86 % 89 % 63 % 92 % Marseille 100 % 89 % 100 % 100 % Leeuwarden 72 % 44 % 88 % 75 % Liverpool 66 % 44 % 63 % 75 %

Bremen 72 % 67 % 63 % 75 % Zagreb 97 % 89 % 88 % 100 % 's-Hertogenbosch 66 % 44 % 100 % 50 % Edinburgh 66 % 44 % 63 % 75 %

Hannover 76 % 78 % 63 % 75 % Rijeka 86 % 78 % 100 % 75 % Amersfoort 76 % 44 % 100 % 75 % Manchester 86 % 78 % 63 % 100 %

Nuremberg 86 % 89 % 63 % 92 % Osijek 79 % 78 % 75 % 75 % Maastricht 79 % 56 % 100 % 75 % Bristol 69 % 56 % 63 % 75 %

Bochum 76 % 78 % 63 % 75 % Split 83 % 78 % 88 % 75 % Oslo 69 % 67 % 25 % 92 % Nottingham 72 % 67 % 63 % 75 %

Weimar 69 % 67 % 50 % 75 % Pula 69 % 78 % 75 % 50 % Stavanger 55 % 56 % 25 % 67 % Brighton and Hove 66 % 44 % 63 % 75 %

Karlsruhe 86 % 89 % 63 % 92 % Budapest 100 % 89 % 100 % 100 % Bergen 55 % 56 % 25 % 67 % York 69 % 56 % 63 % 75 %

Mainz 72 % 67 % 63 % 75 % Pécs 86 % 78 % 100 % 75 % Leiden 76 % 44 % 100 % 75 % Dundee 66 % 44 % 63 % 75 %

Mannheim 83 % 78 % 63 % 92 % Debrecen 86 % 78 % 100 % 75 % Warsaw 100 % 89 % 100 % 100 % Norwich 69 % 56 % 63 % 75 %

Heidelberg 72 % 67 % 63 % 75 % Szeged 86 % 78 % 100 % 75 % Łódź 86 % 78 % 100 % 75 %

Copenhagen 97 % 89 % 100 % 92 % Győr 83 % 67 % 100 % 75 % Kraków 100 % 89 % 100 % 100 %

Note: cities in red have not been ranked due to poor data coverage (<33 % on 'Cultural Vibrancy' or 'Creative Economy'); cities in light blue have not been ranked because outside EU countries.


Table A3. Number of winsorised values and skewness and kurtosis after winsorisation

Sub-indices Dimensions Indicators Winsorised values

Skewness < 2

Kurtosis < 3.5



1. Sights & landmarks 2 1.8 3.6

2. Museums & art galleries 1 1.6 2.6

3. Cinemas 1 1.9 4.5

4. Concert & music halls 2 2.0 4.8

5. Theatres 3 1.8 4.4


6. Tourist overnight stays 4 2.0 4.2

7. Museum visitors 3 1.7 2.9

8. Cinema attendance 2 1.4 2.2

9. Satisfaction with cultural facilities

0 0.0 – 0.7

2. Creative Economy


10. Jobs in arts, culture & entertainment

2 0.7 0.8

11. Jobs in media & communication

1 1.5 3.3

12. Jobs in other creative sectors 0 1.9 6.2


13. ICT patent applications 9 2.1 3.2

14. Community design applications

2 2.0 5.7


15. Jobs in new arts, culture & entertainment enterprises

0 2.0 4.7

16. Jobs in new media & communication enterprises

5 1.9 2.9

17. Jobs in new enterprises in other creative sectors

2 2.0 3.9



18. Graduates in arts & humanities

1 1.5 2.4

19. Graduates in ICT 3 1.5 2.3

20. Average appearances in university rankings

2 1.4 2.5


21. Foreign graduates 3 1.7 2.6

22. Foreign-born population 0 0.9 1.7

23. Tolerance of foreigners 0 0.3 – 0.4

24. Integration of foreigners 0 1.3 1.1

25. People trust 0 0.6 – 0.8


26. Accessibility to passenger flights

2 1.6 2.7

27. Accessibility by road 0 0.7 8.7

28. Accessibility by train 3 2.1 4.8


29. Quality of governance 0 – 0.3 – 1.1

Endnotes | 23

Endnotes

1 https://composite-indicators.jrc.ec.europa.eu/cultural-creative-cities-monitor2 The analysis was based on the recommendations of the OECD–JRC Handbook on Constructing Composite

Indicators (OECD¬JRC, 2008) and on more recent research from the JRC. JRC audits of composite indicators and scoreboards, conducted by the JRC upon request of indices’ developers, are available at: http://composite-indicators.jrc.ec.europa.eu/

3 In order to make 2017 and 2019 scores fully comparable, this new approach has also been used to recalculate missing data from the 2017 edition.

4 Cities having quite different population sizes, GDP per capita or employment rates may ultimately belong to the same group. The use of the continuous value instead of the reference group that a city belongs to allows more precise imputations to be made.For Swiss cities, for which GDP per capita in purchasing power standards is not available, the Luxembourg GDP per capita has been used, as they share the same ‘outlier’ character compared to all other European cities.

5 Groeneveld and Meeden (1984) set the criteria for absolute skewness at above 1 and for kurtosis at above 3.5. The skewness criterion was relaxed to account for the small sample (179 cities). See: Groeneveld, R. A., & Meeden, G. (1984). Measuring Skewness and Kurtosis.TheStatistician,33(4), 391–399. https://doi.org/10.2307/2987742

6 The association between Foreign-born population and D3.2, Openness, Tolerance & Trust, is 0.47, while theother four indicators under D3.2 correlate at 0.55-0.75. This weaker correlation is actually in line with theliterature on the topic, which argues that greater diversity does not necessarily correspond to greater tolerance(Wessel, 2009). Additionally, the association between Cinema attendance and D1.2, Cultural Participation & Attractiveness, is 0.39, while the other three indicators under D2.1 correlate at 0.54-0.71. Again in this case, theresult may be explained by the different composition of cinema audiences compared to tourism and museumvisitors. For a more comprehensive discussion on the selected indicators and suitable improvements. See:Wessel, T. (2009). Does diversity in urban space enhance intergroup contact and toler-ance?GeografiskaAnnaler:SeriesB,HumanGeography,91(1), 5–17. https://doi.org/10.1111/j.1468-0467.2009.00303.x ; Montalto, V., Moura,C. J. T., Langedijk, S., & Saisana, M. (2019). Culture counts: An empirical approach to measure the cultural andcreative vitality of European cities. Cities, 89, 167-185.

7 The indicators Cinema attendance and Foreign-born population also correlate equally or slightly better withother dimensions than the one to which they have been assigned (D3.2, Openness, Tolerance & Trust, and D1.2, Cultural Participation & Attractiveness, respectively), based on both correlation analysis and conceptual relevance. For more details on the development of the Monitor conceptual framework see: Montalto, V., Moura,C. J. T., Langedijk, S., & Saisana, M. (2019). Culture counts: An empirical approach to measure the cultural andcreative vitality of European cities. Cities, 89, 167-185.

8 In the budget allocation method, experts are given a budget of N points to be distributed over a number ofindicators (or dimensions), allocating more to those indicators whose importance they want to stress. Thebudget allocation method can be divided into four different phases: (a) selection of experts for the valuation; (b)allocation of budget to the indicators; (c) calculation of the weights; (d) iteration of the budget allocation untilconvergence is reached (optional). See more at: https://composite-indicators.jrc.ec.europa.eu/?q=10-step-guide/step-6-weighting

9 The four city groups based on population are as follows: ‘XXL’ – more than 1 million; ‘XL’ – between 500 000 and 1 million; ‘L’ – between 250 000 and 500 000; ‘S-M’ – between 50 000 and 250 000.

10 Regarding the aggregation formula, decision theory practitioners have challenged the use of simple arithmetic averages on conceptual grounds because of their fully compensatory nature, in which a comparatively high advantage on a few dimensions can compensate for a comparative disadvantage on many dimensions. Despite justification for the arithmetic averaging formula in the development of the C3 Index, as discussed in the previous section, the geometric average was initially considered as a possible alternative. This is a partially compensatory approach that rewards cities with similar performance in all dimensions; it motivates those cities with uneven performance to improve in those dimensions in which they perform poorly, and not just in any dimension. However, as the geometric average runs counter to the idea of a ‘specialisation’ strategy, which encourages a city to improve in those dimensions where it already has a comparative advantage, it was finally not included in the simulations. A geometric average would contradict a principle adopted in the development of the C3 Index, whereby weak performance in some of the C3 dimensions should not be penalised.

11 Note that the Cultural and Creative Cities Monitor 2019 includes 190 cities, roughly 90 % of the European cities which have been designated, under different approaches, Cultural and Creative Cities. Thirteen cities have been included in the Monitor but not in the final rankings because they did not meet the data coverage criterion, meaning at least 45 % data coverage at the index level and at least 33 % for the ‘Cultural Vibrancy’ and ‘Creative Economy’ sub-indices, or because they were located in countries outside the EU (namely Switzerland and Norway). The rankings and the analysis presented henceforth are therefore always based on a total of 179 cities, but qualitative information is provided for the full sample of 190 cities on the Cultural and Creative Cities Monitor Online.

KJ-04-19-003-EN-N

© European Union, 2019

ISBN 978-92-76-08812-7 doi:10.2760/902078

https://composite-indicators.jrc.ec.europa.eu/cultural-creative-cities-monitor

http://composite-indicators.jrc.ec.europa.eu/

http://composite-indicators.jrc.ec.europa.eu/

https://doi.org/10.2307/2987742

https://doi.org/10.2307/2987742

https://doi.org/10.1111/j.1468-0467.2009.00303.x

https://composite-indicators.jrc.ec.europa.eu/?q=10-step-guide/step-6-weighting

https://composite-indicators.jrc.ec.europa.eu/?q=10-step-guide/step-6-weighting

annex c: statistical assessment of the cultural and creative...

Documents