1

Comparison of CASBEE-City assessment results and citizen

satisfaction with cities

Speakers:

Kawakubo, S.1; Ikaga, T.

2; Murakami, S.

3;

1 Hosei university, Tokyo, Japan

2 Keio university, Yokohama, Japan

3 Institute for building environment and energy conservation, Tokyo, Japan

Abstract: The Comprehensive Assessment System for Built Environment Efficiency (CASBEE-

City) is a tool specifically designed for comprehensive city-scale assessment. CASBEE-City

was first published in Japan in 2011 and a revised version was published in 2012. This study

examines the effectiveness of the CASBEE-City tool by statistically examining the relation

between results of an objective CASBEE-City assessment, which was based on public

statistical information, and a subjective assessment, which was carried out by a nationwide

questionnaire survey of citizen satisfaction. Regression analysis reveals clear relations

between the subjective and objective assessment results. Scatter plots, with objective data on

the horizontal axis and subjective data on the vertical axis, were produced for each CASBEE-

City indicator. Most of the results showed a high coefficient of determination (between 0.5

and 0.8), which shows the tool’s effectiveness.

Keywords, CASBEE, sustainability assessment of cities, questionnaire survey, public

statistical information, citizen satisfaction

1. Introduction

Assessing the performance of municipalities is important for local governments in order to

understand the actual conditions of their cities, towns, and villages (hereinafter,

municipalities) and to assist in determining effective measures for sustainable development.

Many assessment indicators and tools for municipalities have been developed to support these

goals. The International Organization for Standardization (ISO) has also established a

technical committee to gather information about tools for urban sustainable development [1].

These assessment indicators and tools provide valuable information and facilitate

understanding of the actual local conditions. However, most of these indicators and tools are

published without confirming their effectiveness or comparing their actual and projected

performance. It is essential to verify these tools from an academic perspective to accelerate

sustainable development at the city level. Therefore, this study aims to examine the

effectiveness of one city-scale assessment tool and to judge whether it is of practical use.

2. Study method

This study focuses on the Comprehensive Assessment System for Built Environment

Efficiency (CASBEE) tool, also known as “CASBEE for Cities” (hereinafter referred to as

CASBEE-City). The tool was designed specifically for city assessment and developed in

close cooperation with the central government and local governments in Japan. The tool

supports local governments and other stakeholders in identifying environmental, social, and

2

economic characteristics of their cities and in quantifying the effectiveness of city policies.

Development of the initial version of the tool began in 2008 and it was released in 2011 after

extensive discussion among experts in the field [2][3]. A second, revised version was released

in 2012 offering improvements developed by incorporating suggested revisions [4]. One

result of these improvements is the organization of the assessment items as shown in Table 1.

Table 1. Assessment items of CASBEE-City Major and minor categories Sub-categories Assessment indicators Unit

Qu

alit

y (

Q)

Q1. Environmental aspects

Q1.1 Nature conservation Q1.1.1 Ratio of green and water spaces (Forest areas + major lake areas)/

total land area %

Q1.2 Local environment

Q1.2.1 Atmosphere quality

Number of days in which the hourly

concentration of photochemicals during

the day is 0.12 ppm or higher (average)

day

Q1.2.2 Water quality 75% of average daily biochemical oxygen

demand (BOD) in rivers mg/l

Q1.3 Resource recycling Q1.3.1 Recycling rate of general waste Recycling rate of general waste %

Q1.4 CO2 sinks Q1.4.1 CO2 absorption by forests (Current forest area ·unit of absorption)/

adjusted population t-CO2/person

Q2. Social aspects

Q2.1 Living environment

Q2.1.1 Quality of housing Total floor area per dwelling unit m2

Q2.1.2 Traffic safety Number of traffic accidents/

adjusted population

number/

1,000 people

Q2.1.3 Crime prevention Number of crimes recorded/

adjusted population

number/

1,000 people

Q2.1.4 Disaster preparedness Number of disaster response hospitals per

100,000 persons

number/

100,000 people

Q2.2 Social services

Q2.2.1 Adequacy of education services (Number of students/number of teachers)

at elementary and junior high schools -

Q2.2.2 Adequacy of cultural services (Number of community centres + number

of libraries)/land area of municipality number/10 km2

Q2.2.3 Adequacy of medical services Number of doctors/adjusted population number/

1,000 people

Q2.2.4 Adequacy of childcare services Number of childcare facilities/

pre-school population (aged 0– 4 years)

number/

100 people

Q2.2.5 Adequacy of services for the

elderly

Number of senior care facilities/

senior population (aged ≥65 years)

number/

1,000 people

Q2.3 Social vitality

Q2.3.1 Rate of population change due to

births & deaths

(Number of births – number of

deaths)/total population %

Q2.3.2 Rate of population change due to

migration

(Number of move-ins – number of move-

outs)/total population %

Q3. Economic aspects

Q3.1 Industrial vitality Q3.1.1 Amount equivalent to

gross regional product

(Agricultural output + value of

manufactured goods shipments + sales of

commercial goods)/adjusted population

1,000,000

Yen/person

Q3.2 Financial viability Q3.2.1 Tax revenue

Tax revenues of the local

government/adjusted population

10,000

Yen/person

Q3.2.2 Outstanding local bonds Real debt service ratio %

Q3.3 Emissions trading Q3.3.1 Emissions trading Presence or absence of an

emissions trading scheme -

Env

iro

nm

enta

l lo

ad (

L) L1. CO2 emissions from energy sources

L1.1 Industrial sector CO2 emissions from industrial sector/adjusted population t-CO2/person

L1.2 Residential sector CO2 emissions from residential sector/adjusted population t-CO2/person

L1.3 Commercial sector CO2 emissions from commercial sector/adjusted population t-CO2/person

L1.4 Transportation sector CO2 emissions from transportation sector/adjusted population t-CO2/person

L2. CO2 emissions from non-energy sources

L2.1 Waste disposal sector

and others CO2 emissions from waste disposal sector and others/adjusted population t-CO2/person

Note: Adjusted population is calculated as (nighttime population + daytime population) / 2

3

A feasibility study on CASBEE-City was conducted by applying the tool to a nationwide

municipality assessment in Japan [5]. Figure 1 shows the results of the assessment in terms of

the objective environmental, social, and economic conditions of whole municipalities in

Japan. Municipalities with darker shading have better assessment results compared with

municipalities with lighter shading. The questions that are likely to arise here are as follows.

Do citizens in cities with good assessment results feel comfortable about their cities? Do

citizens in cities with bad assessment results demand improvements in city performance? To

answer these questions, a questionnaire survey targeting citizens throughout Japan was

conducted and the collected subjective data were compared with the objective assessment

results delivered by the tool.

70-80%

60-70%

50-60%

40-50%

30-40%

20-30%

10-20%

0-10%

Results for Q1Good

Poor

90-100%

80-90%

(0.61 ≤ x < 0.67)

(0.54 ≤ x < 0.61)

(0.49 ≤ x < 0.54)

(0.43 ≤ x < 0.49)

(0.38 ≤ x < 0.43)

(0.31 ≤ x < 0.38)

(0.24 ≤ x < 0.31)

(0.06 ≤ x < 0.24)

(0.75 ≤ x < 0.92)

(0.67 ≤ x < 0.75)

70-80%

60-70%

50-60%

40-50%

30-40%

20-30%

10-20%

0-10%

Results for Q2Good

Poor

90-100%

80-90%

(0.53 ≤ x < 0.55)

(0.51 ≤ x < 0.53)

(0.49 ≤ x < 0.51)

(0.47 ≤ x < 0.49)

(0.44 ≤ x < 0.47)

(0.42 ≤ x < 0.44)

(0.39 ≤ x < 0.42)

(0.25 ≤ x < 0.39)

(0.59 ≤ x < 0.72)

(0.55 ≤ x < 0.59)

70-80%

60-70%

50-60%

40-50%

30-40%

20-30%

10-20%

0-10%

Results for Q3Good

Poor

90-100%

80-90%

(0.65 ≤ x < 0.72)

(0.59 ≤ x < 0.65)

(0.51 ≤ x < 0.59)

(0.44 ≤ x < 0.51)

(0.38 ≤ x < 0.44)

(0.31 ≤ x < 0.38)

(0.23 ≤ x < 0.31)

(0.02 ≤ x < 0.23)

(0.80 ≤ x < 0.99)

(0.72 ≤ x < 0.80)

Figure 1. Assessment results of all local governments in Japan. Top: Q1(environmental aspects); middle: Q2

(social aspects); bottom: Q3(economic aspects).

4

An online questionnaire survey was conducted during 20–26 November 2012. The

questionnaire was widely distributed across Japan to residents who were at least 15 years old.

To prevent data bias, the questionnaire was carefully distributed to males and females in a

wide age range. Individuals were divided into six groups by gender and age. The

questionnaire was also carefully distributed to municipalities of all different sizes across

Japan. Municipalities were divided into three groups by taking their administration level

(determined by the local autonomy law of Japan) into account. Large cities include

government ordinance cities and the 23 special wards of Tokyo. Medium cities include core

cities and special cities. Small municipalities include other small towns and villages.

Respondents were asked to list their satisfaction level with their municipality for each aspect

(based on the subcategory items of the CASBEE-City tool). Satisfaction level was provided

on a 100-point scale with 0 points representing extreme dissatisfaction, 50 points representing

a neutral option, and 100 points representing complete satisfaction.

Approximately 100 responses were collected from each municipality so that the results for

average citizen satisfaction obtained on the survey would be representative. In total, 12,630

valid responses were collected from the survey, as shown in Table 2.

Table 2. Overview of collected response

Age (years) Number of respondents (%)

Male / Female Size of municipalities Number of respondents (%)

15-29 1,180 / 1,260

(9.3%) (10.0%) Large 2,261 (17.9%)

30-49 2,517 / 2,556

(19.9%) (20.2%) Medium 8,101 (64.1%)

≥50 2,552 / 2,565

(20.2%) (20.3%) Small 2,268 (18.0%)

Total 12,630 (100%) Total 12,630 (100%)

Regression analysis was conducted to quantify the relation between citizen satisfaction level

and the actual performance of municipalities. The average satisfaction level was also

calculated for each assessment item for each municipality so that we can identify which

municipalities have succeeded or failed in satisfying their citizens. Scatter plots, with

objective data on the horizontal axis and subjective data on the vertical axis, were also

produced for each CASBEE-City indicator to visualize those relations.

3. Results

By regression analysis, clear relations were found between the subjective data (level of citizen

satisfaction with their municipalities) and the objective assessment results from the CASBEE-

City tool. Table 3 shows the results of the regression analysis for each assessment item. Most

of the results showed a high coefficient of determination (between 0.5 and 0.8), which

verified the tool’s effectiveness. Figure 2 shows examples of scatter plots produced. It is

clearly demonstrated that citizens who live in better circumstances, as measured on the

CASBEE-City tool, tend to have higher satisfaction levels with their municipalities.

5

Table 3. Results of regression analysis by each assessment item Major

category Minor category Sub-category Estimated regression equation

R2

value

Qu

alit

y (

Q)

Q1

.

Env

iro

nm

en

tal

asp

ects

Q1.1 Nature conservation Q1.1.1 Ratio of green and water spaces y = 0.245x + 44.9 0.681

Q1.2 Local environment Q1.2.1 Atmosphere quality y = 64.7e-0.108x 0.626

Q1.2.2 Water quality y = -10.8ln(x) + 61.8 0.610

Q1.3 Resource recycling Q1.3.1 Recycling rate of general waste y = 0.951x + 25.6 0.741

Q1.4 CO2 sinks Q1.4.1 CO2 absorption by forests y = -1.31x2 + 11.8x + 37.2 0.479

Q2

. S

oci

al a

spec

ts

Q2.1 Living environment

Q2.1.1 Quality of housing y = 0.387x + 9.49 0.744

Q2.1.2 Traffic safety y = -21.6ln(x) + 75.6 0.753

Q2.1.3 Crime prevention y = -22.7ln(x) + 94.3 0.776

Q2.1.4 Disaster preparedness y = 9.28ln(x) + 45.9 0.621

Q2.2 Social services

Q2.2.1 Adequacy of education services y = -20.4ln(x) + 95.5 0.274

Q2.2.2 Adequacy of cultural services y = 7.42ln(x) + 45.0 0.724

Q2.2.3 Adequacy of medical services y = 16.4ln(x) + 37.9 0.587

Q2.2.4 Adequacy of childcare services y = 27.4x + 27.4 0.576

Q2.2.5 Adequacy of services for the elderly y = 15.0ln(x) + 53.7 0.608

Q2.3 Social vitality

Q2.3.1 Rate of population change due to

births & deaths y = 19.5x + 46.0 0.636

Q2.3.2 Rate of population change due to

migration y = -4.38x2 + 9.03x + 49.9 0.530

Q3

.

Eco

no

mic

aspec

ts Q3.1 Industrial vitality Q3.1.1 Amount equivalent to GRP y = 15.6ln(x) + 20.1 0.680

Q3.2 Financial viability Q3.2.1 Tax revenue y = -0.0211x2+2.57x+11.8 0.633

Q3.2.2 Outstanding local bonds y = -23.5ln(x) + 103 0.705

Q3.3 Emissions trading Q3.3.1 Emissions trading － －

Total Quality Score y = 0.952x - 2.76 0.554

Good

Poor

Cit

ize

n s

ati

sfa

ctio

n

(Su

bje

ctiv

e d

ata

)

Value of indicator (Objective data) GoodPoor((Forest areas + major lake areas) / Total land area) [%]

y = 0.245x + 44.9

R² = 0.6810

10

20

30

40

50

60

70

80

90

100

0 20 40 60 80 100

A single plot represents a municipality

Good

Poor

Cit

ize

n s

ati

sfa

ctio

n

(Su

bje

ctiv

e d

ata

)

Value of indicator (Objective data) PoorGood(Number of crimes recorded / adjusted population) [number / 1,000 people]

y = -22.7ln(x) + 94.3

R² = 0.7760

10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30 35

Good

Poor

Cit

ize

n s

ati

sfa

ctio

n

(Su

bje

ctiv

e d

ata

)

Value of indicator (Objective data) GoodPoor(Amount equivalent to gross regional products) [1,000,000 Yen / person]

y = 15.6ln(x) + 20.1

R² = 0.6800

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40

Good

Poor

Cit

ize

n s

ati

sfa

ctio

n

(Su

bje

ctiv

e d

ata

)

Value of indicator (Objective data) GoodPoor(Total Q score of CASBEE-City) [-]

y = 0.9518x - 2.756

R² = 0.5539

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Figure 2. Relation between CASBEE-City assessment result and citizen satisfaction (top left: survey result vs.

ratio of green and water spaces; top right: survey result vs. crime prevention; bottom left: survey result vs.

amount equivalent to gross regional product; bottom right: survey result vs. total Q score from CASBEE-City)

6

4. Conclusions

The study examined the effectiveness of the CASBEE-City tool by collecting citizen

satisfaction data and conducting regression analysis to quanity the relation between citizen

satisfaction and the objective performance of municipalities in Japan. The results show that

there is a high corelation between the subjective assessment (the level of citizen satisfaction

with their municipalities) and the objective assessment (the assessment indicator values of

CASBEE-City), which provides evidence of the tool’s effectiveness. This is the main finding

of the study.

The next step for this line of study is to quantify the influence rate (i.e., the weights) of each

assessment item on the overall citizen satisfaction level to further accerelate the transitions to

a sustainable society. There may be other background factors that affect the citizen

satisfaction level but are not included in the tool at this moment, and it is highly important to

identify those factors. These are future challenges to be solved in the next steps of the study.

References

[1] ISO (2012). ISO Technical Committees TC268. Sustainable development in

communities. [online] (Available at:

http://www.iso.org/iso/iso_technical_committee?commid=656906). (accessed on 10

April 2013).

[2] Murakami, S., Kawakubo, S., Asami, Y., Ikaga, T., Yamaguchi, N., and Kaburagi, S.

(2011). Development of a comprehensive city assessment tool: CASBEE-City. Building

Research and Information, 39 (3): 195-210.

[3] The Committee for the Development of an Environmental Performance Assessment

Tools for Cities (2011). CASBEE for Cities Technical Manual (2011 Edition). Tokyo.

Japan Sustainable Building Consortium (JSBC).

[4] The Committee for the Development of an Environmental Performance Assessment

Tools for Cities (2012). CASBEE for Cities Technical Manual (2012 Edition). Tokyo.

Japan Sustainable Building Consortium (JSBC).

[5] Kawakubo, S., Ikaga, T., and Murakami, S. (2011). Nationwide assessment of city

performance based on environmental efficiency. International Journal of Sustainable

Building Technology and Urban Development, 2 (4): 293–301.