evaluation report - mlhd
TRANSCRIPT
1 | P a g e
Evaluation Report
Canteen Menu Support Program
Project Duration: September 2015 – September 2017
Health Promotion Service
2 | P a g e
For further Information contact:
Alexandra Walker
Acting/ Health Promotion Coordinator | Health Promotion
Integrated Care and Partnerships Directorate
Tumut Community Health, Simpson Street Tumut NSW 2720
Tel 02 6947 0906| Fax 02 6947 0915| Mob 0477 378 888|
DOCUMENT CONTROL
Name of Document: Murrumbidgee Local Health District Canteen Menu Support Program
Section Health Promotion
Date revised December 1, 2017
Supersedes Nil
Endorsed by Christine May, Manager Health Promotion
Contact Alexandra Walker
Author: Natalie McDermott, Health Promotion Officer; Jessica Poll, Health
Promotion Officer; Alexandra Walker; A/ Health Promotion Coordinator.
Reviewed by: Carly Barnes
Acknowledgements: Schools that participated in the Murrumbidgee Local Health District (MLHD) Canteen Menu Support Program for making the collaboration possible. Canteen staff for sharing their wealth of knowledge with the Health Promotion Service; their hard work and commitment to student health and wellbeing. Hunter New England Local Health District for their guidance, and sharing their experiences and evidence around canteen support. Healthy Kids Association for their assistance and support in conducting menu assessments. Health Promotion Service for the quality and enthusiastic support provided to project sites. Natalie McDermott and Jessica Poll for their excellent contribution to the present evaluation. Judy Callinan for creating the infographic to present this report. Kim Gilchrist for her statistical advice and guidance. Christine May for her vision to provide additional support to school canteens and strategic guidance.
3 | P a g e
Table of Contents
Abbreviations ........................................................................................................................ 4
Executive Summary .............................................................................................................. 5
Introduction ........................................................................................................................... 6
Method .................................................................................................................................. 7
Participants ........................................................................................................................... 8
Results .................................................................................................................................. 9
Quantitative results ............................................................................................................ 9
Qualitative results ............................................................................................................ 11
Discussion .......................................................................................................................... 18
Quantitative Evaluation .................................................................................................... 18
Qualitative Evaluation ...................................................................................................... 19
Revision of the School Canteen Strategy ........................................................................ 20
School Changes .............................................................................................................. 21
Limitations ....................................................................................................................... 21
Implications for Future Practice ....................................................................................... 22
Conclusion .......................................................................................................................... 23
References ......................................................................................................................... 23
Appendix 1 – Expression of Interest Letter .......................................................................... 25
Appendix 2 – School canteen pre-commencement survey .................................................. 28
Appendix 3 – HPS Protocol ................................................................................................. 30
Appendix 4 – Feedback Report ........................................................................................... 32
Appendix 5 – Action Plan .................................................................................................... 34
Appendix 6 – School Canteen Post-Project Survey ............................................................. 35
Appendix 7 – HPS Post-Project Survey .............................................................................. 38
Appendix 8 – Canteen Menu Support Program Infographic ................................................ 40
4 | P a g e
Table of Figures
Figure 1. Overall number of items each assessment. .......................................................... 10
Figure 2. Compliance according to FT@S Strategy ............................................................ 10
Figure 3. Overall Compliance .............................................................................................. 11
Figure 4. Role of respondent ............................................................................................... 11
Figure 5. Avenues of support provided ................................................................................ 12
Figure 6. School community supportive of changes ............................................................ 12
Figure 7. Will the canteen changes be maintained? ............................................................ 13
Figure 8. Confidence levels ................................................................................................. 13
Figure 9. Project Team Confidence Pre- and Post-Project .................................................. 14
Figure 10. FT@S Strategy level of Understanding .............................................................. 15
Figure 11. Most Useful Tools .............................................................................................. 15
Figure 12. Suggestions provided during action plan sessions ............................................. 16
Figure 13. Strong executive involvement in menu changes ................................................. 17
Figure 14. Enjoyment .......................................................................................................... 17
Abbreviations
DoE Department of Health
Fresh Tastes Fresh Tastes @ School NSW Healthy School Canteen Strategy
HKA Healthy Kids Association
HNE LHD Hunter New England Local Health District
HP staff Health Promotion Staff
LLWatS Live Life Well @ School
MLHD Murrumbidgee Local Health District
MoH Ministry of Health
NSW New South Wales
OPH Office of Preventative Health
PHIMS Population Health Information Management System
RCT Randomised Controlled Trial
WHO World Health Organisation
5 | P a g e
Executive Summary
In 2015, the overweight and obesity rate for NSW children was 22% and has remained
relatively stable since 2007. In 2008, in an effort to educate children in healthy eating habits
and ultimately reduce the risks associated with poor physical health, the NSW Ministry of
Health, in partnership with the NSW Department of Education, introduced the Live Life Well
@ School (LLWatS) program. Since its introduction, 83% of primary schools, including central
schools in the Murrumbidgee Local Health District (MLHD) have completed LLWatS training.
In 2012, desirable practices were introduced to guide implementation. One of the 10 desirable
practices focussed on the school food environment and expected school canteens to adhere
to the Fresh Tastes @ School NSW Healthy School Canteen Strategy (Fresh Tastes) that was
introduced in 2004.
In 2016, invitations were extended to all schools across MLHD with 21 schools of varying sizes
becoming involved in the MLHD Canteen Menu Support Program.
The objective of this project was to increase the proportion of schools in the MLHD that
complied with the by June 2017. This was to be achieved through (1) building the capacity of
the Health Promotion staff (HP Staff) working with schools to support positive menu change
in food services; and (2) by inviting schools to participate in a menu audit and feedback
process including goal setting, action planning and HP staff implementation support to each
school each school term for 12 months.
A multi-strategic intervention was used to improve adoption of Fresh Tastes in primary,
including central schools. Menu assessments were conducted over four terms with results
showing a 12.86% increase in green items, a 6.32% decrease in amber items and a 5.22%t
decrease in red items. Overall compliance with Fresh Tastes increased from 0% to 29.41%.
The program evaluation demonstrated that through intensive support, school canteens were
more likely to become compliant with Fresh Tastes. It is reasonable to expect that with
continued support from DoE, canteen menu changes will continue under the revised strategy
that replaced Fresh Tastes in 2017.
6 | P a g e
Introduction
In 2015 in NSW, the overweight and obesity rate in children was 22% and has remained
relatively stable since 2007 (Bravo et al., 2016). In an effort to educate children in healthy
eating habits and ultimately reduce the risks associated with poor physical health, NSW
Ministry of Health (MoH), in partnership with the Department of Education (DoE), introduced
the Live Life Well @ School (LLWatS) initiative. Since the introduction of the initiative in 2008,
83% of primary schools, including central schools, in the Murrumbidgee Local Health District
(MLHD) have completed LLWatS training (NSW Ministry of Health, 2017).
As part of the LLWatS initiative, desirable practices were introduced in 2012. Of the 10
desirable practices, one looks at the food environment and expects school canteens to adhere
to the Fresh Tastes @ School NSW Healthy School Canteen Strategy (Fresh Tastes) that was
introduced in 2004. Fresh Tastes followed the principles of the Australian Guide to Healthy
Eating and also classified food into three categories:
“RED ‘Occasionally’ – Do not sell these foods on more than two occasions per term.
AMBER ‘Select carefully’ – Do not let these foods dominate the menu and avoid large
serve sizes.
GREEN ‘Fill the Menu’ – Encourage and promote these foods in the canteen” (NSW
Department of Health & NSW Department of Education and Training, 2012, pg. 5).
Fresh Tastes encouraged canteens to sell more than 50% of green foods and less than 50%
of amber foods (NSW Ministry of Health & NSW Department of Education and Training, 2012).
Across NSW in 2014, less than 30% of primary and secondary schools were compliant with
Fresh Tastes (Woods et al, 2014). In MLHD, between April and June 2015, 47.37% of schools
were achieving desirable practice number 5 (NSW Ministry of Health, 2017). However, this
number included schools that do not have canteens but meet the other two components of the
practice.
Woods et al (2014) reported that of the red food items sold regularly on the school canteen
menu, cakes and biscuits, ice-creams, crisps, chocolate, confectionary and pastries tended to
sell. It was also reported that small, lower socio-economic schools, non-government and rural
schools were more likely to have a higher number of red items on the canteen menu (Hills et
al, 2015).
Woods et al (2014, pg. 113) stated that “the nutrition policy for school canteens is supported
by Health and/ or Education Department guidelines or policy, so it is surprising to find such
low levels of compliance.” The ‘Nutrition in Schools Policy’ states that reviews should occur at
least every two years by the Principal and monitoring should occur by the school education
directors (NSW Department of Education and Training, 2011).
Challenges or barriers identified for reducing the likelihood of school canteens meeting the
Strategy include:
Multiple competing interests such as parental views and student food preferences,
support from the families (Woods et al, 2014; Ardzejewska, Tadros & Baxter, 2012,
Wolfenden et al, 2014).
Organisational objectives such as financial benefits (Woods et al, 2014; Ardzejewska,
Tadros & Baxter, 2012).
7 | P a g e
Complex and time-consuming administrative procedures (Woods et al, 2014).
Limited resourcing of policy implementation (Woods et al, 2014).
Complex nature of the guidelines – including confusion over packaged products, and
classification of products (Woods et al, 2014; Ardzejewska, Tadros & Baxter, 2012),
and
The role of the wider community (Ardzejewska, Tadros & Baxter, 2012).
In March 2017, the revised NSW Healthy School Canteen Strategy Food and Drink
Benchmark was launched jointly by DoE and MoH. The Food and Drink Benchmark aims to
be implemented in all NSW public schools by the end of 2019. Catholic and Independent
schools are strongly encouraged to adopt the strategy. The revised strategy aligns with the
Australian Dietary Guidelines but also incorporates the Health Star Rating national labelling
system. The minimum food and drink benchmark applies to school canteens and vending
machines. It requires each menu section to consist of at least 75% ‘Everyday’ food and drinks
and no more than 25% ‘Occasional’ items with a Health Star Rating of at least 3.5. Portion
limits also apply and promotion and marketing practices are enforced.
Woods et al (2014) found that a way to facilitate change was through ongoing support by
monitoring and evaluating the canteen menu. The Randomised Controlled Trial (RCT)
conducted by Hunter New England Local Health District (HNELHD), implementation between
November 2014 and April 2015, supported this approach by offering a multi-strategic, audit
and feedback intervention to enhance implementation of the healthy canteen policy. This
support was provided to the intervention schools through policy implementation, executive
support, consensus processes, training, tools and resources, academic detailing, recognition,
performance monitoring and feedback and marketing strategies (Wolfenden et al, 2017).
Research conducted by Nathan et al (2012, pg. 656) found “that the multi-strategic intervention
involving leadership support, staff training and telephone follow-up support, was effective in
increasing the uptake of …[everyday foods]… compared to more minimal, information-based
support.
The objective of this project was to increase the proportion of schools in the MLHD that comply
with the Fresh Tastes by June 2017. This objective was to be achieved through: (1) building
the capacity of HP staff working with schools to support positive change in food services, and
(2) by inviting schools to participate in a menu audit and feedback process including goal
setting, action planning and implementation support by HP staff each term for 12 months.
Method In September 2015 the MLHD Health Promotion Service initiated the development of the
MLHD Canteen Menu Support Program with a thorough and collaborative needs assessment.
The Audit and Feedback method developed by HNELHD RCT provided best evidence of
practice in the school canteen setting. The MLHD Health Promotion Service adapted the
implementation according to local needs and service capacity levels.
Key documents were developed to implement the project plan and included a protocol,
canteen menu assessment guidelines, a menu assessment tool, a menu review feedback
report template and a menu improvement action plan template. Other documents used to
8 | P a g e
complete a menu assessment included the Fresh Tastes @School Canteen Menu Planning
Guide, Tool Kit and Menu Analysis Assistant, the HNELHD Menu Analysis Assumptions for
non-participating schools, Good for Kids Good for Life Canteen Product Search and Healthy
Kids Association NSW (HKA) School Canteen Buyer’s Guide.
All schools in the MLHD region were invited to submit an expression of interest for an
opportunity to be placed among 20 schools in the MLHD Canteen Menu Support Program.
The program ran over four school terms with menus reviewed each term. HP staff reported
back by the end of each term and facilitated an action planning process to implement menu
improvements. At the end of the first term and after submission of an action plan, schools
received kitchen equipment vouchers for the value of $500.
For the initial term, intensive support was available to HP staff from the project leader but the
level of support decreased over time as capacity building took effect. During the
implementation of the project, three menus from canteen project sites were reviewed by the
entire team as part of continued capacity building efforts. Upon launch of the revised canteen
strategy in March 2017, an additional five menus were reviewed by the team to increase
knowledge, skills and confidence in menu reviews under the new criteria.
Team capacity building included a one day training facilitated by a canteen operations expert
and six team menu review activities. Menu reviews were completed in pairs and results
discussed via teleconference. This process triggered the need to develop tailored resources
to facilitate assessments.
A menu review required approximately one hour of work and another hour to discuss results
in a team setting via teleconference. In total it is estimated that at least 34 hours were invested
in capacity building for HP staff (six hour training session, plus 14 menu reviews taking
approximately two hours each).
After the launch of the revised strategy menu reviews occurred following both Fresh Tastes,
to ensure consistency for the project, and the revised strategy to assist project schools to
begin meeting the revised criteria. Thus, a new and improved menu assessment tool was
developed together with a portion size easy guide and a team consensus document.
Participants
There were 22 schools that originally expressed interest in being involved in the MLHD
Canteen Menu Support Program, two of which did not meet the inclusion criteria thus were
excluded from the project. Of these schools:
One was a special education school and the relevant HP staff would work with them
outside the scope of the project.
Another school was excluded as it received lunch orders through the local shop and
therefore did not meet the selection criteria.
Following commencement of the project another school withdrew due to significant
changes.
As the project progressed a further two schools opted out of the project, however two
schools were invited to take their place.
9 | P a g e
The demographics of the schools were as follows:
57% (n=12) Department of Education primary schools.
24% (n=5) Catholic primary schools.
14% (n=3) Department of Education central schools.
5% (n=1) Independent primary school.
67% (n=14) schools were Inner Regional.
24% (n=5) schools were Outer Regional.
9% (n=2) schools were Remote or Very Remote.
38% (n=8) schools were identified as disadvantaged (Quintile 2).
33% (n=7) schools were identified as most disadvantaged (Quintile 1).
29% (n=6) schools were neither advantaged nor disadvantaged (Quintile 3).
43% (n=9) schools with between 160 and 300 students.
24% (n=5) schools with between 26 and 159 students.
19% (n=4) schools with between 301 and 450 students.
14% (n=3) schools with between 451 and 700 students.
Results
Quantitative results A Population Health Information Management System (PHIMS) report extracted in September
2016 indicated that 47% of project sites were deemed achieving Practice 5 of LLWatS. Table
1 shows the nine project schools deemed compliant with Practice 5 were in fact, after their
initial menu assessment, not complaint with Fresh Tastes
Table 1. Schools deemed achieving practice 5 in PHIMS (07/09/16) when their menu indicates lack of compliance to FT@S Strategy.
School Initial assessment % Green
Initial assessment % Amber
Initial assessment % Red
Initial assessment banned
Grey items*
School A 25.00% 70.80% 4.20% 0.00% 0.00%
School B 55.03% 35.19% 9.79% 0.00% 0.00%
School C 17.00% 57.00% 26.00% 0.00% 0.00%
School D 46.00% 48.00% 6.00% 0.00% 0.00%
School E 53.00% 43.00% 4.00% 0.00% 0.00%
School F 36.00% 57.00% 7.00% 0.00% 0.00%
School G 34.90% 53.50% 11.60% 0.00% 0.00%
School H 58.30% 35.40% 6.30% 0.00% 0.00%
School I 74.00% 21.00% 0.00% 0.00% 5.00% * Food and drink were classified as grey items when HPS were unsure of category.
To complete menu reviews each item on the menu was counted. Counting included the same
item but in the different serving size e.g. a 350ml bottle of water and a 600ml bottle counted
as two items but excluded the same item served hot or cold e.g. cheese sandwich and a
toasted cheese sandwich. Over the four assessments menus showed a 10% decrease in
canteen menu items (Figure 1).
10 | P a g e
Figure 1. Overall number of items each assessment (Ax).
Initial menu assessments indicated food items as 43.47% green, 46.47% amber, 8.37% red,
1.31% banned, and 0.37% grey. Following the final assessment menus showed a 12.86%
increase in green items, a 6.32% decrease in amber items and a 5.22% decrease in red items
(Figure 2).
Figure 2. Compliance according to the Fresh Tastes @School Strategy
880.00
900.00
920.00
940.00
960.00
980.00
1000.00
1020.00
1040.00
Items on menu
Items on menu Ax2
Items on menu Ax3
Items on menu Ax4
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Initial Assessment 2nd Assessment 3rd Assessment 4th Assessment
Green
Amber
Red
Banned
Grey
11 | P a g e
Figure 3 shows that assessments conducted each term resulted in an increase in compliance
with Fresh Tastes. The final assessment resulted in 29.41% (n=5) of schools being compliant.
Figure 3. Overall Compliance
Qualitative results
Perceptions from Project sites To evaluate the success of the project from the school perspective all 19 schools involved
were sent a post-project evaluation survey. Of these 19 schools, 17 (89.5%) participated in
the project for the entire duration with two schools being invited to join after the
commencement of the project.
Of the 59 people who were sent the survey, 15 responded, a return rate of 25%. Respondents
included canteen managers (40%, n=6), school principals (26.67%, n=4), teachers and P&C/
P&F member (both at 13.33%, n=2) as seen in Figure 4.
Figure 4. Role of respondent
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
Compliance Ax1 Compliance Ax2 Compliance Ax3 Compliance Ax4
SchoolPrincipal
CanteenManager
Canteenvolunteer
Teacher P&C /P&Fmember
Other
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
12 | P a g e
As part of the project, various forms of support were provided. Figure 5 shows how useful the
respondents found the different avenues. The visits from the local HPs staff was seen as the
most beneficial form of support (64.24%, n=9) followed by the menu review report (57.14%,
n=8).
Figure 5. Avenues of support provided
Few respondents found the HKA canteen workshop beneficial with 57.14% (n=8) responding
that it was not applicable. On examining the attendance list for the Wagga Wagga canteen
workshop it was noted that 68% (n=13) of schools involved with the project did not attend.
With changes happening within the canteen, it was important to identify if these changes were
accepted by the school community. All respondents believed the majority of students and
teachers had been supportive of the changes (Figure 6) while one respondent (7%) indicated
that the majority of families had not been supportive of the changes.
Figure 6. School community supportive of changes
Me
nu
re
vie
w r
ep
ort
Can
teen
Men
uIm
pro
vem
en
t A
ctio
nP
lan
Vis
it f
rom
loca
l HP
S
Emai
l an
d t
ele
ph
on
eco
nta
ct b
y lo
cal H
PS
Can
teen
Wo
rksh
op
he
ldin
Wag
ga in
No
vem
ber
20
16
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
Very beneficial
Beneficial
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Yes No
Students
Familes
Teachers
13 | P a g e
It was also felt by 92.31% (n=12) of respondents that the canteen changes would be
maintained, as indicated in Figure 7. A school commented that it was also important for the
canteen to run at a profit while another respondent felt that the revised strategy “will be a little
onerous.”
Figure 7. Will the canteen changes be maintained?
An outcome of the project was to increase confidence in those involved with the canteen menu
development to provide food and drinks that were healthier choices. Of the respondents,
92.31% (n=12) felt that their confidence had increased to improve the canteen menu to include
everyday items (Figure 8). A comment made by a respondent indicated that they felt confused
over the guidelines given that sugar filled milk and juice drinks were approved for sale while
foods containing sugar were not and that the guidelines were conflicting at times.
Figure 8. Confidence levels
Yes No
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Yes No
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
14 | P a g e
A final question asked how the Health Promotion Service could assist the schools better.
Comments were made by four respondents and included:
Continuing to provide support and information.
Supplying healthy alternative suggestions and recipes.
Working with suppliers and manufacturers to ensure products available meet the
revised guidelines, and
Setting up a Facebook group where schools can share ideas and discuss what does
and does not work; primary and secondary schools to be separate.
Perceptions of Health Promotion Staff To evaluate the success of the project from the perspective of the Health Promotion Service,
all eleven HP staff involved in the project were sent a post project evaluation survey. Of these
82% (n=9) completed the survey.
HP staff were asked to rate their confidence for pre and post project (Figure 9). Prior to the
commencement of the project, the majority of HP staff (44.44%, n=4) indicated they were
slightly confident to conduct a menu review and 44.44% (n=4) felt confident to facilitate change
in school canteens (44.44%, n=4). Confidence levels grew as the project was implemented
with most indicating that post project they felt very confident to conduct a menu assessment
(44.44%, n=4) and more than half felt very confident to facilitate change in school canteens
(55.56%, n=6).
Figure 9. Project Team Confidence Pre- and Post-Project
Prior to the commencement of the project, 77.78% (n=7) of HP staff indicated they understood
some components of Fresh Tastes. As the project concluded 66.67% (n=6) indicated they had
full understanding. (Figure 10).
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Very confident Confident Slightly confident Not confident Unsure
Confidence Pre-Project to conduct menu assessments
Confidence Post-Project to conduct menu assessments
Confidence Pre-Project to facilitate change
Confidence Post-Project to facilitate change
15 | P a g e
Figure 10. FT@S Strategy level of Understanding
Various tools were available to support HP staff during the project with the felt usefulness
shown in Figure 11. The ‘Good for kids good for life’ online canteen product search by HNE
LHD and the Fresh Tastes @School Canteen Menu Planning Guide were seen as the most
useful tools (77.78%, n=7). Additionally, one HP staff commented that calling another HP staff
was also a useful tool.
Figure 11. Most Useful Tools
HP staff were asked how menu assessments were affected as schools became more
compliant with Fresh Tastes. Responses were grouped thematically and recorded below.
Difficult knowing how to progress and keep schools interested and wanting to make
change.
Easier, positive and developing relationship that didn’t exist.
Confidence, knowledge and skill of the canteen staff and HP staff increased which
contributed to a shared vision towards improvement of the canteen menu, and
How well did youunderstand the FT@SStrategy prior to thecommencement of
the project?
How well do youunderstand the FT@SStrategy now as theproject draws to an
end?
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
Full understanding
Understand some components
No understanding
Unsure
FT@S Menuanalysis tool
by HKA
HKAtelephone
support
Good forkids Goodfor life -online
canteenproduct
search byHNE LHD
HKA buyersguide
Menuanalysis
assumptionstool by HNE
LHD
FT@SCanteen
MenuPlanning
Guide
MLHDcanteen
menuassessmentguidelines
0.00%10.00%20.00%30.00%40.00%50.00%60.00%70.00%80.00%90.00%
16 | P a g e
Some principals were wanting to become compliant and doing so by achieving the
percentage although green foods weren’t good sellers.
Action plans were to be completed following each menu assessment as part of the protocol,
however these were received by 100% of schools in the initial term only. During development
of the action plans HP staff were to restrain from providing suggestions, ensuring the school
felt empowered and retained ownership. Suggestions were only to be provided if the school
requested assistance or were struggling to produce their own actions.
Suggestions were provided by 66.67% (n=6) of HP staff during the action plan session (Figure
12). Suggestions:
Were provided always where needed.
Were provided only if asked, and
Facilitated the discussion so canteen staff could decide their priorities and draft the
action plan based on the items in the menu that were non-compliant.
Figure 12. Suggestions provided during action plan sessions
Greater than two thirds (66.7%, n=6) of HPS reported there was a strong executive
involvement in menu changes (Figure 13). Comments included:
Some project sites do and some don’t, and
Principals haven’t been involved in the action planning process but were supportive of
the canteen manager decisions.
Yes No
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
17 | P a g e
Figure 13. School executive involvement in menu changes
HP staff were asked to provide feedback on their enjoyment of the project. Of the nine HP
staff who completed the survey, eight (88.88%) provided comments for this question. At the
commencement of the canteen work 25% (n=2) of HP staff said they enjoyed all components,
compared to 37.50% (n=3) enjoying it all at the end of the project (Figure 14).
Figure 14. Health promotion staff enjoyment
Comments included:
Enjoying it less since the revised strategy.
Too confusing and with not having all the resources available when the revised
strategy was launched, therefore making it hard to complete a menu assessment.
Depends on the site and their level of enjoyment with committed sites making the
experience positive, and
It was a grind as they felt their canteen project sites were not all that engaged in the
process.
Yes No
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
Enjoyed it all Enjoyed somecomponents
Didn't enjoy
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
How much were youenjoying the canteenwork at projectcommencement?
How much are youenjoying the canteenwork now?
18 | P a g e
When asked how the project implementation could be improved, comments included:
Developing a menu assessment tool at the start of the project to ensure a uniform and
systematic approach.
Provide the gift vouchers at the end of the project.
Practise canteen operation as per Serena Copley (Trainer and Assessor at the
Australian College for Higher Studies) training.
HP staff perception survey to be done at project commencement.
Canteen newsletter and more celebration of successes.
Timely support for sites following project implementation plan.
Contingency plans to facilitate problem solving strategies to cope with unexpected
problems/ conflicts involving canteen staff, and
The Project Management System (an in-house task completion spreadsheet) might
have helped keep people on track as people forgot about milestones and to consult
more with stakeholders to check that the intervention was acceptable even though
HNE evidence suggested it was worthwhile.
Lastly, HP staff were asked to suggest ideas on how canteens could be further supported.
Ideas suggested included:
Jumping at chances as they arise and continuing to have a presence in canteen
settings.
Ensure public school canteens are aware that the revised strategy is mandatory and
to reassure that there is plenty of time for incremental changes.
Include consumers in development of the project, build on relationships we have
already developed with principals, canteen staff and P&Cs, and
Networking is the only feasible way due to limited time we have to invest so much
intense support (some sites need intense support though).
Discussion
The aims of this project were to assist schools in complying with Fresh Tastes and increase
the confidence of those involved in the schools canteens to improve the nutritional value on
the menu.
In this project there were two forms of evaluation:
1) Quantitative – which evaluated the actual changes in the canteen menus measured
against Fresh Taste requirements and
2) Qualitative – which evaluated the confidence of those involved in the project e.g. canteen
managers and volunteers, HP staff and school principals; along with the usefulness of
support and resources provided. The evaluation also identified how accepting the school
community was of the changes occurring in the canteen.
Quantitative Evaluation
Prior to the commencement of the project a PHIMS report indicated that 47% of the project
sites deemed achieving Practice 5 of LLWatS were in fact, not achieving this practice.
Desirable practices are assessed by HP staff during 6 to 12 month scheduled follow-up visits
with schools trained in LLW@S. This inaccuracy of schools achieving Practice 5 demonstrates
the limitations of the scheduled follow-up visit with the school as an opportunity to gather
19 | P a g e
detailed information to make an accurate assessment of the menu compliance to Fresh
Tastes. This finding also strengthens the need to complete menu reviews to appropriately
assess Practice 5.
Similar to the Woods et al (2014) findings, the results from this project reflected predominantly
amber items with green items measuring 43.47%. Of the red items sold in canteens from the
project schools, the most common items were: Dim Sims, hot dogs, muffins, pies, and sausage
rolls. These also reflect the findings in Woods et al (2014). Similarly, the project schools in
disadvantaged areas (n=7) support the findings in Hills et al (2015) of selling red item foods.
As part of the canteen menu assessments, nutrition information panels were examined and
checked to see how the food product fitted with the nutritional values indicated on the Fresh
Tastes Menu Analysis. An issue with some canteen menus was that the food item came from
the local bakery and there was no nutritional information. This created a challenge by not being
able to clearly identify if a food item was amber or red, thus assumptions were made and led
to possible inaccuracies on the menu assessment. Wolfenden et al (2014) states that a barrier
to change is the lack of skills by canteen key stakeholders to classify products. This supports
the challenge found by HP staff where nutritional information was not available.
Throughout the project, canteens showed a 12.86%, 6.32% and 5.22 % change in green,
amber and red items respectively. These results were comparable with those achieved in the
HNE LHD RCT where results showed a 15%, 8.8% and 5% percentage point change in green,
amber and red items respectively after 12 to 14 months of support (Wolfenden et al, 2017).
Qualitative Evaluation
A barrier or challenge listed by Woods et al (2014), Ardzejewska, Tadros & Baxter (2012), and
Wolfenden et al (2014) was the perceived lack of support from the school community. From
the schools that participated in the project there was just one school that indicated the families
were not supportive of the change; the remaining schools indicated that the whole school
community was supportive of the changes occurring in the school canteen thus disproving
these above mentioned findings.
On asking the respondents how the Health Promotion Service could assist those involved with
the school canteen better, a response was through continuing to provide support and
information. This supports the findings by Woods et al (2014) which states that a facilitator of
change is by building support through educating key partners and stakeholders along with
ongoing, high quality training and assistance. These findings were also supported by this
project where the most valued, or beneficial, support provided was the visits by the local HP
staff.
When asked about level of confidence, one respondent stated that they felt confused over the
guidelines while another found the guidelines conflicting at times. Both these comments
support the findings by Ardzejewska, Tadros & Baxter (2012). It was indicated by Woods et al
(2014, pg. 113) that a barrier to change is the “complex nature of the guidelines.” Despite this
barrier and the findings by Ardzejewska, Tadros & Baxter (2012), the majority of schools felt
the changes would be maintained, however, one school commented that it was also important
for the canteen to run at a profit which also supports the findings by Woods et al (2014).
20 | P a g e
Research shows that strong executive support (Ardzejewska, Tadros & Baxter, 2012) is
required to ensure policy compliance. From the project schools, HP staff indicated that 66.67%
of schools had a strong executive support. Given the surveys were anonymous it is difficult to
identify if the schools with strong executive support became compliant, or close to compliant.
Woods et al (2014) states that a facilitator of policy implementation was the institutional
champions. A challenge in this project was that nine schools had a change of champion; this
was either through the canteen manager leaving, the principal moving to another school or
the key contact leaving the school.
Nathan et al (2012) states that to be effective as a multi-strategic intervention, stakeholders,
or canteen champions, first provide thoughts and suggestions on how to make change;
support staff need to facilitate the stakeholders instead of prescribing suggestions. The HP
staff involved in this project supported this approach by providing suggestions and assistance
only when invited.
Feedback received from HP staff as canteen menu assessments were completed was the lack
of time they had to provide feedback. HP staff were advised to notify the school that the
feedback session could take two hours, however, a challenge was fitting the meeting in with
the key stakeholders. Some visits occurred between recess and lunch thus limiting the time
available to provide feedback and allow the meetings to be relaxed and not rushed.
Revision of the School Canteen Strategy
The NSW Healthy School Canteen Strategy Food and Drink Benchmark has been developed
to align with the latest Australian Dietary Guidelines and incorporates the Health Star Rating
national front of pack labelling system. The strategy has four elements, three relating directly
to food and the other relating to marketing and upselling the healthier items. There are now
two categories: Everyday foods and Occasional foods.
Everyday foods are the foods belonging to the five food groups, which are: vegetables and
legumes/ beans, fruit, grain (cereal) foods such as bread, rice, pasta and breakfast cereals
(choose mostly wholegrain and/ or high fibre varieties), lean meats and poultry, fish, eggs and
tofu, milk, yoghurt and cheese and alternatives (choose mostly reduced fat varieties).
Everyday drinks include water, milk, and 99% fruit juice. Everyday foods should fill at least ¾
(75%) of each section of the menu with a range of these healthy foods and drinks from the
five food groups or meals made from them.
Occasional foods and drinks are those foods which are high in saturated fat, sugars and/ or
salt and often have little nutritional value. Only the healthiest versions of these foods (Health
Star Rating of 3.5 stars and above) should be sold in school canteens. Occasional foods
should be limited to no more than ¼ (25%) of each section of the menu.
The revised strategy also incorporates portion limits for Everyday flavoured milk, juices and
hot meals and all Occasional foods and drinks.
The strategy encourages Everyday foods to be priced favourably, promoted and advertised
with the aim of making Everyday foods more noticeable and the popular choice. Occasional
foods should not be promoted or displayed in prominent positions.
21 | P a g e
The introduction of the revised strategy in March 2017 posed some challenges in determining
the effectiveness of the level of support provided to school canteens. Of the schools
participating in the project, 59% (n=10) declined a menu assessment in Term one. A reason
provided by four schools for this was owing to wanting an assessment with the revised strategy
and the canteen managers wanting to see what the changes were going to be in regards to
approved items.
School Changes Four schools disengaged from the project at various stages for reasons including lack of
sustainability of canteen operation relying on teachers, wrong perception of achieving strategy
compliance, and unresolved conflict between canteen stakeholders.
School U joined the project late as a result of another school opting out. Upon the initial
assessment the menu was deemed not compliant with Fresh Tastes (23% green, 60% amber,
14% red and 3% banned) but after three terms of academic detailing the menu became
compliant (60% green and 40% amber).
Limitations
Two main operational limitations were found in the project. Firstly, as the project was
implemented more detailed information around food and drink items became available. A slight
margin of error was identified in the categorisation of some items. A decision to not amend
errors was made in the interest of ensuring consistency in data and enabling it to capture
menu review changes. Secondly, project site and HP staff surveys were not conducted before
the project so qualitative results rely on post-project survey data.
Fresh Tastes was superseded by the revised NSW Healthy School Canteen Strategy at
project mid-point. This triggered the need for team capacity building and development of tools
which extended the duration of the project for a whole school term.
Recommendations
1. Due to sites being academically detailed in each of the four terms, scheduling
appointments became challenging in Term 4 and Term 2. Assessments to be conducted
on a six monthly basis following the change between summer and winter menus.
2. Academic detailing included a menu review report followed by the development of an
action plan. Menu review reports alone were proven ineffective thus action plans are to
be written to accompany the menu report.
3. Executive approval of Menu Improvement Action Plans facilitated menu improvements.
School visions should align with the World Health Organisation (WHO) Health Promoting
Schools Framework.
4. Rapport building with canteen stakeholders and their empowerment was identified as a
precursor to menu improvements. Therefore relationships with canteen stakeholders
need to be developed.
22 | P a g e
5. During capacity building of HP staff to complete menu reviews, an array of methods were
provided including an unprotected menu assessment spreadsheet. Over time HP staff
experienced difficulties in their calculations as the spreadsheet became modified and
unreliable. An effective tool to be developed to ensure quality of reports delivered to
schools.
6. At times, to improve the menu, HP staff needed to provide suggestions to canteen staff
about food and drink items beyond what is recommended in the HKA Buyer’s Guide. Thus
it is imperative to work with local food suppliers and build their knowledge.
7. Some canteens produce homemade food items. The provision of recipe modification
became apparent. The development of recipe cards will support the provision of healthy
homemade food items.
8. Project sites received a kitchen equipment gift voucher after completion of the initial
assessment and action plan, however throughout the project few action plans were
received following assessments. For this reason gift vouchers are to be provided at the
end of the project to encourage continued project site engagement.
9. The development of a communication strategy is required to include a canteen project
newsletter, celebration of successes through media releases, as suggested in the
feedback provided by HP staff
Implications for Future Practice
As the MLHD Canteen Menu Support Program draws to an end by school term 3 2017, the
Health Promotion Service has been actively reflecting upon lessons learnt and the available
capacity to continue to support school canteens. Three main constraints for future work have
been identified:
1. DoE accepted greater responsibility for the implementation of the NSW Healthy School
Canteen Strategy and the full nature of this support will be provided in 2017.
2. Office of Preventative Health (OPH) is currently developing a menu assessment service
for schools with accompanying resources for school and HP staff use.
3. Target group capacity building is needed to ensure sustainability of strategies and this
has proven to require large investments in time. We acknowledge the limitations of
providing multi- strategic implementation support to a large number of schools within
existing staff capacity.
With the consideration of the above constraints the Health Promotion Service is developing a
continuous practice improvement project titled “Canteen Networks: Building Capacity in
School Canteens”. The aim is to establish four school canteen networks, with 70%
participation by canteen stakeholders, to build capacity to achieve Practice 5 of the LLWatS
program in the MLHD region, by the end of April 2018.
Five school canteen menus became compliant with Fresh Tastes over four terms and by the
end of the project. Upon assessment under the revised strategy, no menu was found
compliant. The establishment of canteen networks will enable canteen staff to become part of
a supportive environment that will facilitate their ability to become compliant with the revised
strategy.
23 | P a g e
Conclusion
This project used a multi-intervention approach to provide support to school canteens in order
to ensure compliance with Fresh Tastes. This approach resulted in a variety of outcomes:
confidence of canteen key stakeholders increased in ensuring a higher number of everyday
items were available on the school canteen menu, the HP staff felt more confident in
conducting menu assessment reviews and providing support and assistance to canteen key
stakeholders, and there was an increase in compliance of school canteens meeting the
requirements of Fresh Tastes.
Through the introduction of the revised school canteen strategy, avenues for further
development have been identified including confirmation of the responsibility of the DoE to
ensure schools are meeting the canteen requirements.
Greater support is required to assist rural and remote schools and schools in disadvantaged
areas to provide everyday foods through the school canteen.
References Ardzejewska, K.,Tadros, R., & Baxter, D. (2012). A descriptive study on the barriers and facilitators to implementation of the NSW (Australia) Healthy School Canteen Strategy. Health Education Journal, 72(2), 136-145.
Bravo, A., Innes-Hughes, C., O’Hara, B. J., McGill, B., & Rissel, C. (2016). Live Life Well @ School: Evidence and Evaluation Summary 2008 – 2015. NSW Ministry of Health, North Sydney.
Hills, A., Nathan, N., Robinson, K., Fox, D., & Wolfenden, L. (2015). Improvement in primary school adherence to the NSW Healthy School Canteen Strategy in 2007 and 2010. Health Promotion Journal of Australia, 26, 89-92.
NSW Department of Education and Training. (2011). Nutrition in Schools Policy (policy number PD/2011/0420/V01). Retrieved October 11, 2013, from https://education.nsw.gov.au/policy-library/policies/nutrition-in-schools-policy Nathan, N., Wolfenden, L., Bell, A. C., Wyse, R., Morgan, P. J., Butler, M., Sutherland, R., Milat, A. J., Hector, D., & Wiggers, J. (2012). Effectiveness of a multi-strategy intervention in increasing the implementation of vegetable and fruit breaks by Australian primary schools: a non-randomised controlled trial. BMC Public Health, 12, 651-659.
NSW Department of Health & NSW Department of Education and Training. (2012). Fresh Taste @ School: NSW Healthy School Canteen Strategy. Canteen Menu Planning Guide (3rd ed.). North Sydney.
NSW Government of Health. Population Health Intervention Management System (PHIMS). Accessed 1st June, 2017.
Wolfenden, L., Nathan, N., Williams, C. M., Delaney, T., Reilly, K. L., Freund, M., Gillham, K., Sutherland, R., Bell, A. C., Campbell, L., Yoong, S., Wyse, R., Janssen, L. M., Preece, S., Asmar, M., & Wiggers, J. (2014). A randomised controlled trial of an intervention to increase the implementation of a healthy canteen policy in Australian primary schools: study protocol. Implementation Science, 9, 147-53.
Wolfenden, L., Nathan, N., Janssen, L. M., Wiggers, J., Reilly, K., Delaney, T., Williams, C. M., Bell, C., Wyse, R., Sutherland, R., Campbell, L., Lecathelinais, C., Oldmeadow, C.,
24 | P a g e
Freund, M., & Yoong, S. L. (2017). Multi- strategic intervention to enhance implementation of healthy canteen policy: a randomised controlled trial. Implementation Science, 12, 6-16.
Woods, J., Bressan, A., Langelaan, C., Mallon, A., & Palermo, C. (2014). Australian school canteens: menu guideline adherence or avoidance? Health Promotion Journal of Australia, 25, 110-115.
28 | P a g e
Appendix 2 – School canteen pre-commencement survey
Survey of Canteen Managers 2015
The Fresh Tastes @ School NSW Healthy School Canteen Strategy provides direction to all schools about the type and amount of food to be supplied through school canteens to support healthy eating and good health.
Canteen Managers are key to making these healthy food choices easier for children. However, this can be a challenging task so the Health Promotion Service of the Murrumbidgee Local Health District would like to know how it can better support you in your work.
We invite you to complete the following survey. Your feedback will assist us to develop strategies, tools and resources to support healthy eating in school canteens in the district in 2016.
School Information: 1. School name (for your chance to win a $250 kitchen equipment voucher):
____________________ 2. Please tick which category fits your school.
Department of Education
Catholic Independent Primary Central
3. Number of students enrolled: __________________ 4. Distance, in kilometres, to nearest major fresh food outlet: ______________
Canteen Information:
5. Does your school have: Canteen Food source outside school (e.g. corner store taking lunch orders)
6. Who is responsible for selecting food and drinks for the canteen menu? (tick all the apply)
Principal
P&C LLW@S Teacher
Canteen Committee
Other: ______________
7. Number of days the canteen is open or outside food source is available: 1 2 3 4 5
8. Number of canteen paid staff: ______________ 9. Hours paid per day: ___________ 10. Number of canteen volunteers: ___________ 11. What is the purpose of your canteen?
Provide a food service To generate profit Not sure Other (please specify): ________
12. Who are your main food suppliers?
13. Are you aware of the NSW Nutrition in Schools policy?
Yes No 14. Are you aware of the Fresh Tastes @ School NSW Healthy School Canteen
Strategy?
29 | P a g e
Yes No
15. Do you know if your school canteen or outside food source complies with the Fresh Tastes @ School NSW Health School Canteen Strategy?
Yes No 16. What is the top selling item on your canteen menu?
(a) Food ___________ (b) Drink ________ 17. What is your biggest barrier to introduce new green foods in your menu? e.g:
children won’t buy it, lack of profit, not enough time to prepare it:
Training and Development 18. Have you attended any school canteen training in the past 3 years?
Yes No If no, please explain why not: _________________________________________________
19. Would you be interested in attending training in 2016? Yes No
20. If yes, please tick the topic of training: (tick all that apply) Making your canteen profitable Understanding Fresh Tastes policy Menu development Healthy eating Food safety Marketing and promotion Cooking Supporting volunteers Other: (please specify) __________________________________________________
21. Does the canteen link with the classroom teaching/ school kitchen garden?
Yes No Unsure
If yes, please give an example:
22. Has your school menu ever been reviewed by your school LLW@S committee, an officer from the Health Promotion Service, the NSW Healthy Kids Association, Dietitian or other?
Yes No Unsure
23. Is your school a member of NSW Healthy Kids Association?
Yes No Unsure
24. Have you met your local Health Promotion Officer?
Yes No Unsure
25. Is there anything else you want to share with us? Please use the reverse side of this page.
Thank you for completing this survey, please return to: Maree Oddy, Albury Area Office, Suite 1B, PO Box 3095, Albury NSW 2640 [email protected]
30 | P a g e
Appendix 3 – HPS Protocol Guidelines for Health Promotion Practitioners to implement the canteen project.
Step 1- each HPS to obtain a copy of the canteen menu for their project schools.
Step 2- call the canteen contact person as listed in the EOI submitted by each school. Ideally we want to have an electronic copy of the menu so we can colour code it and attach it to the final report. If you acquire further contact details from the school please update them on PHIMS.
Step 3- once the menu is received, decide if it should be reviewed individually, in pairs or in a group.
Step 4- screen the menu with the assumptions tool which will generate questions which will enable the elimination of assumptions as much as possible.
Step 5- arrange an appointment (preferably 1h) with the canteen to attain the necessary information about items sold in the canteen and/ or to answer the questions generated from the initial assumptions tool screen. Ask for permission to take photos of the Nutrition Information Panel (NIP) and obtain recipes. Allow time to meet with the Principal if requested.
Step 6- during the visit, build rapport with canteen staff and achieve an understanding of canteen dynamics including but not limited to: school culture, service times (lunchtime, recess), school community readiness for change, canteen history, number and capacity of volunteers, canteen purpose, best and least selling items, canteen manager responsibilities, executive support, financial stresses, recipe sources, best way and times to exchange communications, etc.
o Step 6a- If operational or financial issues arise, mention and offer support from the HP team that will treat all information and documentation under confidentiality and identify strategies to overcome those stresses. Refer to Serena Copley training and HKA Business support if school is member or if it is a small school <100 students.
o Step 6b- re- inforce the project process. A canteen menu feedback report will be presented to the Principal, canteen staff and interested parties in a 2h meeting in term 3. It will be followed by the development of an action plan.
How to review a menu – prior to canteen visit:
a) Determine if you need assistance in the menu review process. Contact Alex Walker (AW) in first instance.
b) Screen items with Assumptions tool. c) Identify questions regarding items.
How to review a menu – after canteen visit:
1) Count items as per rules- if desired use spreadsheet template 2) Use a combination of tools including: HKA menu analysis, Good for Kids Good for Life
canteen database (superseded HNE database) http://www.goodforkids.nsw.gov.au/primary-schools/canteens/product-search/ and our internal consensus sheet: ..\Team menu review activities\Tools\MLHD canteen consensus.docx
3) For items for which we have NIPs please use HKA menu analysis flowchart. 4) For identified red items request recipes if they are homemade. 5) For recipe analysis, consult with AW. 6) Once completed submit report and the colour coded electronic menu to AW for final
check by 08/07/16 (this date will change as project is implemented).
31 | P a g e
Presentation of menu review report: Compile review findings in the canteen menu review feedback report by inserting new values in the pie chart and your contact details.
Identify a top 3 list of recommendations for your own use, not to be disclosed to the school unless they are struggling with their action plan. Arrange a visit with the school (Principal, canteen staff and interested parties) and provide an estimated duration of 2hrs.
At the visit present the report and colour coded menu (colour version- Maree to print out for those without access to colour printing). Ensure they understand the findings are kept confidentially. Explain the meaning of the pie chart and its relationship with FT@S criteria. Reinforce the Nutrition in School
Policy key points: All schools should promote and model healthy eating in their programs and activities when
they relate or involve food or drink.
Fresh Tastes @ School (NSW Healthy School Canteen Strategy) is mandatory for all DoE schools and is endorsed by Catholic and Independent schools.
Sugar sweetened drink ban (>300kj or >100mg sodium per serve). i.e.: soft drinks, energy drinks, fruit drinks, sports drinks, cordial, ice tea and others.
Mandatory PDHPE from K- 10 including nutrition education. Other key learning areas should reinforce healthy eating wherever possible.
School activities on/off site including food and drink should be consistent with Fresh Tastes @ School.
Healthy eating consideration should be given to food and drink offered to the wider community.
Principals are responsible for the implementation of the policy and for the canteen operation.
Allow time for silence and reflection. Encourage questions and discussion around the report, FT@S and key points. Create a safe environment for open and honest discussion. Clarify that if you don’t have the answers you will find them and get back to them.
Make quick notes while the discussion is unfolding of items that may become a goal in the action plan. Present them with the action plan template, explain this will become their document to drive their action. Nominate a scribe within the group and facilitate the discussion.
Don’t place any pressure on the number of goals they agree on.
Once a draft is achieved, ask them to finalise it in an electronic version we will email to them, together with the report and colour coded menu.
After meeting: Send email to key canteen contacts including Principal to include report, colour coded menu and action plan template. Once final action plan is received, approve it, store it in shared drive and send to AW so she can arrange voucher delivery ($500 catering equipment).
Subsequent visits: Repeat the process if the menu is different to the last one and follow the process above in term 4 2016, term 1 2017 and term 2 2017.
If there is no need to review the menu, please review the action plan. If challenges arise at any point in the implementation please discuss with AW.
35 | P a g e
Appendix 6 – School Canteen Post-Project Survey
If they answer ‘Yes’ in Q2 they will continue on to Q3. If they answer ‘No’ they will answer Q11 and then will end the survey.
SURVEY FOR CANTEEN PROJECT SITES
37 | P a g e
SURVEY FOR CANTEEN PROJECT SITES SURVEY FOR CANTEEN PROJECT SITES SURVEY FOR CANTEEN PROJECT SITES