By Dan Schober, Program Evaluator & Founder at Informed Community Health

Editor’s note: This post was originally published on LinkedIn.

“I waited…in anticipation…”please get this” I hoped, as the students worked…”

Across the country, schools are addressing childhood obesity by cooking more from scratch to improve the healthfulness of foods served in their cafeterias, with little evidence that this is actually making a difference.

In 2013, I led an evaluation of LiveWell Colorado’s LiveWell@School Food Initiative (“LW@SFI”). LW@SFI had a team of chefs that trained Colorado food service workers to cook healthy foods, from scratch. Although Colorado is just one state, this initiative involved a wide variety of schools – urban and rural – that share similarities to schools across the country.

The central evaluation measure was the Quantitative Menu Analysis, which involved assessing six components of school meals (1. Protein, 2. Beans/Legumes, 3. Dairy, 4. Grains, 5. Produce, and 6. Sauces) as either: “Processed,” “Transitional” or “Scratch.” This real-world evaluation approach would help LW@SFI understand what foods schools were willing to change and this could inform school cafeterias across the country.

Although using this evaluation tool was no easy task, and involved lengthy interviews with food service directors, we were able to successfully use the Quantitative Menu Analysis with 9 Colorado middle schools. We were excited to see some significant changes in both scratch cooking and the overall healthfulness of foods served to children in these schools.

While my colleagues and I were able to use the Quantitative Menu Analysis measure, I wondered how easily other school food evaluators could use this measure. Although replication of complex evaluation instruments is not often considered – it’s important, especially with programs like the LW@SFI, that train food service workers to prepare healthier foods – something being done across the country, as over 31% of children and youth are either overweight or obese[1].

A Test in the Classroom

This fall, I am teaching a Program Evaluation course to Master of Public Health students. One of the evaluation case studies we discussed was the LW@SFI. Most students in the Program Evaluation course are new to evaluation – and this was my chance to see if new evaluators could use the instrument.

I lead a discussion about the measurement instrument (the Quantitative Menu Analysis) and then presented the class of 14 students, with a scenario:

Please imagine that you are classifying a school lunch that consists of:

  • A bologna sandwich with miracle whip on white bread
  • Steamed potatoes
  • Cottage cheese
  • Canned pears
  • Grape juice

I provided food labels for each item in this example lunch and gave students the Quantitative Menu Analysis rubric (that involved classifying six components of school meals: 1. Protein, 2. Beans/legumes, 3. Dairy, 4. Grains, 5. Produce, and 6. Sauces) as either: “Processed,” “Transitional” or “Scratch”).

Then, I waited…in anticipation…”please get this” I hoped, as the students worked…wondering if they would pick this up…I was ready to jump in if I saw or heard confusion…

To my surprise (and excitement) it was only a matter of minutes until one student looked up, finishing the classification of this meal in just 2 minutes. Then another. Then everyone. Even better, they got it. Quickly. Easily. With confidence!

Implications

In addition to catching on quickly – there was a broad consensus among the students, of classifications for the food items in this meal as “Processed,” “Transitional” or “Scratch,” giving me reasonable hope that schools working to increase scratch cooking (and improve the healthfulness of the foods served in their cafeterias), can use this tool to understand and improve their efforts to cook healthier foods and address childhood obesity. It was a small victory to relish – as a program evaluator and a teacher.

Links and Reference

For more information on this evaluation: http://onlinelibrary.wiley.com/doi/10.1111/josh.12413/abstract

For more information on the LiveWell@School Food Initiative: (https://livewellcolorado.org/healthy-schools/school-food-initiative/

[1] Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States. JAMA. 2014;311(8):806-814.