validity and reliability of food security measures

Ann. N.Y. Acad. Sci. ISSN 0077-8923

ANNALS OF THE NEW YORK ACADEMY OF SCIENCESIssue:Paths of Convergence for Agriculture, Health, and Wealth

Validity and reliability of food security measuresCarlo Cafiero,1 Hugo R. Melgar-Quinonez,2 Terri J. Ballard,1 and Anne W. Kepple11Statistics Division, United Nations Food and Agriculture Organization, Rome, Italy. 2McGill Institute for Global Food Security,School of Dietetics and Human Nutrition, McGill University, Montreal, Quebec, Canada

Address for correspondence: Carlo Cafiero, Statistics Division, United Nations Food and Agriculture Organization, via delleTerme di Caracalla 4, Room C-412, 00153 Rome, Italy. [email protected]

This paper reviews some of the existing food security indicators, discussing the validity of the underlying conceptand the expected reliability of measures under reasonably feasible conditions. The main objective of the paper isto raise awareness on existing trade-offs between different qualities of possible food security measurement toolsthat must be taken into account when such tools are proposed for practical application, especially for use within aninternational monitoring framework. The hope is to provide a timely, useful contribution to the process leading tothe definition of a food security goal and the associated monitoring framework within the post-2015 DevelopmentAgenda.

Keywords: food security; validity of empirical measures; post-2015 Development Agenda

IntroductionMeasurement is indisputably an important elementof the process through which we advance knowl-edge. It is indispensable when we need to highlightchanges such as the progress toward set targets. Tocontribute to knowledge and to allow correct assess-ments, however, measurement should be valid andreliable, posing two fundamental but distinct prob-lems regardingwhat is being measured and how it isdone.

Assessing validity and reliability of measures isparticularlyproblematic in the social sciences,wherethe phenomena we wish to measure are often notdirectly observable. The definition of what is beingmeasured may become confounded with its mea-surement to the point that use of different measureswould imply adoption of different notions of thephenomenon being analyzed, an issue that couldeasily gounappreciatedby less sophisticated readers.Confusion may thus arise, especially when dealingwith complex constructs.

This problem, we argue, has been particularly ev-ident in the academic and political debates on foodsecurity definition and measurement over the lasthalf century. Hundreds of indicators have been pro-posed, ranging from quantification of food supplies

at the country level, to more or less detailed char-acterizations of food consumption at the householdor individual level, and including measures of nu-tritional outcomes with respect to growth and nu-tritional deficiencies. Some of these indicators arepresented as measures of food security, others arecombined in various ways to produce indexes, butquestions remain regardingwhether they can indeedbe considered proper measures.

With such questions inmind, in this paperwedis-cuss a few of the existing indicators of food security.Afterhighlighting someof the elementsof thedebatethat are relevant for the definition of a measurablefood security construct, we introduce a conceptualframework that distinguishes the validity of a mea-surement tool from the reliability of the measures itproduces.We then apply the proposed framework toindicators belonging to two categories: those basedon the concept of food consumption adequacy, andthose that treat food insecurity as a condition thatcan be identified and characterized by experiencesand behavioral responses that appear to be commonacross most cultures. In the first group, we considerthe prevalence of undernourishment (POU, which iscurrentlyusedas theofficial indicator tomonitor thetarget 1C of the firstMillenniumDevelopment Goalto eradicate extreme poverty and hunger), the Food

doi: 10.1111/nyas.12594

230 Ann. N.Y. Acad. Sci. 1331 (2014) 230248 C 2014 New York Academy of Sciences.

Cafiero et al. Validity and reliability of food security measures

Consumption Score (FCS, used by the World FoodProgramme in their food security and vulnerabilityassessments), and the Household Dietary DiversityScore (HDDS, which has been proposed as perhapsthe simplest possible proxy for the adequacy of foodconsumption at the household level).

In the second group, we include a numberof experience-based food security scales, startingfrom the Household Food Security Survey Mod-ule (HFSSM, developed by the United States De-partment of Agriculture and used in the UnitedStates and Canada), and continuing with theHousehold Food Insecurity Access Scale (HFIAS,promoted by the Food and Nutrition TechnicalAssistance-II (FANTA-II) initiative), the harmo-nized Latin American and Caribbean Food Secu-rity Scale (ELCSA, from the Spanish Escala Lati-noamericana y Caribena de Seguridad Alimentaria),and the recently launched Food Insecurity Experi-ence Scale (FIES), developed and adopted by theFood and Agriculture Organization (FAO) in col-laboration with the Voices of the Hungry Project.

We conclude, based on our analysis, that the in-adequacy of food consumption can be reliably mea-sured at the country level as the percentage of thepopulation that does not have access to enough foodto fulfil normal requirements. Proposed metrics offood consumption adequacy at the household orindividual levels, unfortunately, do not possess theanalytic foundations required to determine theirempirical validity as food security measures. In par-ticular, there is no basis to establish the reliabilityof the measures obtained in practice or to ensurecomparability across applications.

Experience-based scales, on the other hand,emerge as very promising tools to measure a validconcept of food insecurity at the household andindividual levels. Their use, especially if coupledwith other indicators of individual or householdsocioeconomic condition or nutritional status,can contribute to a better understanding of thedeterminants and consequences of household andindividual food insecurity.

In our discussion, we also draw conclusions onthe suitability of these indicators as elements of amonitoring system at the global scale, in the hopeof providing a timely, useful contribution to theprocess leading to the definition of the post-2015Development Agenda, where a specific goal on foodsecurity will likely be included.

Toward the definition of a measurableconstruct of food securitya

In the early years, the food problem was funda-mentally identified with having enough supplies tocover the needs of the population.13 It was only af-ter 1974 that the expression food security startedto be widely used, even though mostly still refer-ring to adequacy of country-level food supplies.In the 1980s, the focus shifted to looking at thefood problem from the perspective of peoples ac-cess to food, owing to the growing evidence ofwidespread malnutrition and famines in the devel-oping world, even in periods of relatively abundantfood supplies.4 To mark the difference in these dif-ferent approaches, the qualified expression house-hold food security began to be used, stressing theidea that evidence of food insecurity was to be inves-tigated at the household level. In the 1990s, anothershift occurred from considering food security essen-tially a matter of dietary energy adequacy to includ-ing economic, social, nutritional, and psychologicalconsiderations.5,6 This evolution in thinking aboutfood security has been described as a process char-acterized by the occurrence of important and over-lapping paradigm shifts: (a) from the global andthe national to the household and the individual, (b)froma food-first perspective to a livelihoodperspec-tive, and (c) from objective indicators to subjectiveperception.7

The search for indicators has always been presentin the debate that has accompanied the continuingevolution in thinking about food security, with aproliferation of definitions and hundreds of indica-tors being proposed for different uses.8

It is perhaps with the World Food Summit(WFS)9 and, later, the United Nations MillenniumDeclaration on theMillenniumDevelopment Goals(MDG),10 when globally endorsed, time-bound tar-gets associatedwith theoverarchinggoal to eradicatehunger were set,b that the relevance ofmeasurement

aRetracing the history of food security definitions is be-yond the scope of this paper, and we focus here only onelements that are relevant for measurement. For a thor-ough review of the evolving definition of food security,mainly from the institutional and political perspective,see the volume by Shaw;5 see also Gibson.6bThe WFS set the target to halve, by the year 2015, thenumber of people suffering from hunger. Four years

231Ann. N.Y. Acad. Sci. 1331 (2014) 230248 C 2014 New York Academy of Sciences.

Validity and reliability of food security measures Cafiero et al.

and its relation with the definition of the object be-ing measured emerged in all its importance. Whilethe goal and the targets were framed with respectto hunger, the choice of indicators revealed the dif-ficulty in operationalizing the concept as a mea-surable construct. Estimates of the prevalence andnumber of undernourished individuals, as providedby the FAO,11 and of the percentage of children un-der 5 years of age that are underweight, as providedby UNICEF,12 were adopted as official indicators oftheMDGmonitoring framework.Neither of the twoestimates, however, can be considered a measure ofhunger. Rather, they reflect respectively, the extentof insufficiency of food consumption in a countryand thepresenceof symptomsof childmalnutrition.

In the extensive debate around food securitymea-surement, including that generated from the 2002International Scientific Symposium hosted by theFAO,13 it has clearly emerged that the demand formeasures of food security reflecting the various di-mensions of availability, access, utilization, and sta-bility (as captured by the definition set forth by theWFS in 1996c ,14) has not yet been adequately ful-filled. This ongoing debate has been summarizedin several recent publications where indicators arereviewed.1519

Most notably, the need has emerged to go beyondthe national level when assessing the state of foodinsecurity in a country, owing to the fact that rele-vant differences in developmental achievements areseen more within, rather than between, countries.This consideration, along with the increased statis-tical capacity in many countries, has promoted anincreased use of data from population surveys asopposed to national accounts and macroeconomicdata. Relying more directly on survey data is justi-fied also as a response to another identified need,namely to improve the way the food access dimen-sion is captured at the individual or household lev-els. To respond to the new need, efforts have been

later, the MDG proposed to halve, between 1990 and2015, the proportion of people who suffer from hunger(a much less demanding target in the presence of a fastgrowing population).c Food security, at the individual, household, national,regional and global levels [is achieved] when all people, atall times, have physical and economic access to sufficient,safe and nutritious food to meet their dietary needs andfood preferences for an active and healthy life.

made in several directions. We focus here on twoof these efforts: the improvement of measures offood consumption and the development of direct,experience-based measures of the severity of foodinsecurity, defined as the inability to obtain ade-quate food in quantity and quality because of a lackof money or other resources.

Validity and reliability of household foodsecurity measuresIn choosing the criteria to assess the overall qualityof the indicators we review, we are primarily guidedby the consideration that individual or householdfood security metrics should be effective means tomonitor progress toward the goal of ensuring foodsecurity for all, mindful of the prospect that sucha goal will likely be agreed upon by the interna-tional community when defining a renewed, post-2015 Development Agenda.d

The criteria we apply are those of validity andof reliability (which in turn includes accuracy andprecision) in a framework that partly overlaps withthe one proposed by Frongillo.20

ValidityThe terms validity and validation have been usedwith different meanings in different occasions, andgeneral consensus may still not exist.e ,21

Wikipedia provides a very broad definition of va-lidity as follows: In science and statistics, validity isthe extent to which a concept, conclusion or mea-surement iswell-founded andcorrespondsaccuratelyto the real world.22 We believe this definition is toobroad and that the issues of whether ameasurementis well-founded and whether it is accurate are two

dAt the time this paper was written, the most au-thoritative proposal for a relevant target on food se-curity to be considered in the post-2015 Develop-ment Agenda reads as follows: By 2030, end hungerand ensure that all people have access to adequate,safe, affordable, and nutritious food all year round.(Zero draft of the introduction and proposed goals andtargets on sustainable development for the post-2015Development Agenda, produced by the Open Work-ing Group on Sustainable Development Goals, avail-able at http://sustainabledevelopment.un.org/content/documents/4523zerodraft.pdf).eThe International vocabulary of metrology22 does nothave an entry for the term validity.



separate issues. Knowing whether a measurementtool is inadequate because the underlying theoret-ical conceptual framework is poorly grounded orbecause of the inaccuracy of the measures it pro-duces, may be useful, for example, in guiding re-search aimed at improving them.

With reference to food security measurement, re-searchers often considered criterion validity and con-current validity,23 by suggesting that validation of ameasurement tool could be sought by comparingthemeasures it produces with, respectively, an exist-ing gold standard or with other measures capturingother aspects of the same construct obtained at thesame time on the same objects. However, as food se-curity is a nonmaterial, unobservable construct (i.e.,a latent trait), for which no objective benchmark ex-ists, we cannot strictly rely on criterion validation,while concurrent validationmust rely on the a prioriassumption that the concurring measures are valid.In reviewing validation studies of food securitymea-sures, it is difficult to separate judgment of the ap-propriateness of the underlying conceptualizationof food security from the technical properties of thespecificmeasurement tool being validated. To do so,we need to somehow steer away from criterion andconcurrent validation as the proper means to assessvalidity.23 Thus far, the processes of defining foodsecurity as a measurable construct and of validatingproposed metrics have been so inextricably linkedthat judgments on technical characteristics of the in-dicators have often been confused with judgmentson the appropriateness of the underlying concepts.This has led to two opposite and equally dangerousmistakes: concerns regarding the appropriateness ofthe concept have beenpresented as limitations of themeasures, and acceptance of the appropriateness ofthe concept has been considered sufficient to claimvalidity of the indicators.

For these reasons, and by extending a definitionaccepted in the educational and psychological test-ing domains, we propose that validity be intendedas the degree to which both evidence and theorysupport the interpretations of measures entailed byproposed uses of the tool.2426 This definition hasthe advantage that both theoretical and empiricalconsiderations have a role in assessing validity. Theexplicit need to formulate a model that links the-ory and evidence in a probabilistic setting, more-over, may force analysts to take statistical aspectsinto proper consideration, which, in our opinion,

has been greatly overlooked in most discussions onfood security measurement.

Reliability fA measurement tool is reliable if it consistently pro-duces good results. It is a judgment on the overalldegree of uncertainty that inevitably surrounds themeasures.

Traditionally, reliability has been considered thecombination of two distinct qualities, accuracy (ortrueness) and precision, respectively related to thetwo kinds of errors that may affect a measure. Thefirst is a measure of how small the systematic erroris, while the second is a measure of how small theaccidental (or random) error is.

According to Frongillo, accuracy is assessed byin-depth analysis and by relating the measure to acriterion measure, which may be a more definitivemeasure, determinant or consequence.21 However,as the food insecurity construct is unobservable,there is clearly no definitive direct measure in themore traditional sense. Establishing accuracy musttherefore confront the issue of also considering howthemeasure is related to the indirect evidence on theattribute of interest through a theoretical model.The possibility to assess the presence and the ex-tent of systematic measurement error therefore de-pends on the appropriateness of themodel that linkschanges in the attribute of the object of interest (themeasurand)27 to changes in the observed evidence(the data).

Reliability is important also for another reason,as it implies, for example, invariance of the measureobtained when the tool is applied to the same objectunder different conditions.g In the area of food se-curity measurement, this means that measures have

f A note of caution is appropriate here. The most recentedition of the International Vocabulary of Metrology22

has dropped the term reliability and changed themeaningassociated with the term accuracy, which is defined now asa synonymofwhatwe still call reliability, as the result of thecombination of trueness and precision.Wehave decided tocontinue to use the terms reliability instead of accuracy,and accuracy instead of trueness, as we believe they stillreflect themost common, albeit imprecise, understandingof the concepts.gAs some measurement error is unavoidable, invariancehere must be intended in a probabilistic sense, meaningthat the expected size anddirectionofmeasurement errorsare not affected by the object being measured.



no bias and are similarly precise when applied todifferent populations. It has thus a strict relation tothe possibility ofmeaningfully comparingmeasuresobtained from different population groups within acountry or across countries. Lackingmeans to assessreliability, it is difficult to defend comparability.

Beforemoving on, it is worth noting that a funda-mental underlying themewhendiscussingmeasuresis feasibilityintended as the possibility that suffi-ciently reliable and valid measures can be obtainedunder normal conditions. While feasibility is key, aspractically relevant measures should be feasible toproduce, we should not make the mistake of con-sidering a measure to be acceptable only because itis feasible.

A review of selected food securityindicatorsFrom the perspective of the procedure to establishvalidity outlined in the previous section, we dis-cuss indicators on the basis of measures of access tofood. A distinction is made between two categoriesof indicators: those that are based on the concept ofadequacy of food consumption and those that arefounded on the severity of the constraints on foodaccess, also called experience-based food securityindicators.h

Measures of the adequacy of foodconsumptionIt has often been suggested that the extent of foodinsecurity in a population could be conceived asthe proportion of individuals who do not consumeenough food and that it would suffice to take a large

hThere are several examples of indexes and scores (includ-ing someof the oneswe reviewhere) that are commonlybut perhaps too looselypresented asmeasures of house-hold or individual food security. Sometimes also explicitreference is made to the multidimensionality of the foodsecurity concept, hinting that the proposed indicatormight be intended to capture more than one attribute ofhousehold food security, including, for example, the uti-lization dimension. A thorough discussion on the prob-lems with multidimensional measurement is beyond thescope of this paper and we limit ourselves to the analy-sis of validity of the proposed indicators as measures ofthe main attribute they have been designed to capture,namely food access.

sample of people and determine how many arenteating enough to measure it.28

While intuitively appealing, such an approachposes several theoretical and practical challenges. Tostart with, even defining food as a generic measur-able entity presents nontrivial problems. Nutritionstudies have revealed how not only the quantity, butalso thequality and thevarietyof foods andhowtheyare combined, contribute to determining the overallnutritional value of what is consumed. How to de-fine andmeasure the food aggregate for the purposeof assessing its adequacy is thus problematic. Differ-ent solutions have been adopted, leading to differentoperational definitions of food. The most commonsolution is touse thedietary energy content to aggre-gate the quantities of different foods and to conductthe assessment with respect to the resulting totaldietary energy consumption (DEC). Inadequacy isthen defined as when habitual food consumptionfails to provide the amounts of dietary energy re-quired for an active and healthy life,29 a choice thathas been criticized because energy-rich diets may bedeficient of essential nutrients. The energy contentof a certain combination of foods, while measur-able, is not the best way to capture adequacy withregard to nutritional value.i

i The use of total dietary energy as a metric to capture thenutritional adequacy of food has traditionally been justi-fied by the consideration that it would seem improbablethat a dietary intake which is sufficient to cover energyrequirements will be insufficient to meet protein require-ments, except for the possible exception of populationswhich subsist on starchy roots, tubers, plantains, etc., allof which are extremely low in protein content. On theother hand, if a diet is adequate in protein content butthe quantity of food eaten is insufficient to meet energyneeds, some of the protein intake will be used as a sourceof energy and the diet would in practice be deficient bothfor energy and protein requirements.35 More recently,the debate has moved beyond energy and protein consid-eration to include micronutrient adequacy. A meaningfulmeasure of the overall diet that would reflect its nutri-tional value, especially with reference to micronutrients,would require detailed observation on the quantity andquality of food consumed, as well as how it is prepared.Reference to average consumption of individual foods ornutrients, in this case, is likely to be of limited use. Forexample, assessing the quantity of citrus fruits and ofdark leafy vegetables independently does not provide the



A related problem is how to establish adequacyof the food consumed. This is still an active area ofresearch, aimed at reconciling consistency betweennutritional theory and feasibility of data collection.Requirements have been established for individualmacronutrients and of their combination, as wellas for micronutrients, with reference to a series ofparameters linked to sex, age, body mass and com-position, physical activity level, physiological statusof pregnancy or lactation, and to possibly idiosyn-cratic levels of metabolic efficiency.3032 This meansthat adequacy of the food consumed by an individ-ual should be ideally assessed against each personsown specific needs. This in turn would call for de-tailed informationon the characteristics listed aboveto be able to assess the nutritional requirements ofindividuals. A single threshold could be used for agroup only if variability of the requirements withinthe group is known to be very small, possibly smallerthan themeasurement errors that can be expected toaffect the variable that is used tomeasure consump-tion, lest increasing the risk of one of two possi-ble mistakes. Failing to recognize inadequacy whenit is there, or identifying inadequacy of consump-tion when, in reality, it is not there. With the datausually available through household and individualsurveys measuring food consumption, it is impossi-ble to reduce the uncertainty on individual nutrientrequirements to such negligible size as to preventmisclassification. This is why, for example, nutrientrequirements are always specified as ranges of val-ues within a class or population group.j The mainconsequence of this is that the extent of nutrient ad-equacy in a sample of individuals or in a populationcan only be properly assessed in probabilistic terms,which in turn requires defining a model to link theprobability of inadequacy to the observed food con-sumption level and the other relevant characteristicsof the individual.

Another problem relates to the reference periodfor food consumption measurement. Actual foodintake can vary from day to day without serious im-plications for the human body, which has the abilityto store energy and to use it flexibly. It is thus clear

proper evidence on the effective level of iron intake, whichimproves by combining the two foods.j See, for example, UNU/WHO/FAO,31 in particular, page5.

that adequacy of food intake cannot be establishedby considering what is eaten in a day or over a veryshort period. This also has nontrivial implicationsfor data collection and methods of analysis.

In summary, when defining indicators of food se-curity on the basis of food consumption data, onehas to make several choices on various aspects andclearly define the concept of adequacy of food con-sumption so that it is unambiguously understoodbefore even considering how to measure it. Accep-tance of the validity of the concept should then notbe confused with considerations of the feasibility ofreliable assessments.

In the following section, four indicators of theinadequacyof food consumptionwill be considered:the FAOs estimates of the POU, indicators of caloricinadequacy based on food consumption data fromhousehold surveys, dietary diversity, and the FCS.

The FAOs estimates of prevalence ofundernourishmentThe first attempts at measuring the incidence ofundernutrition by considering the distribution offood consumption within populations were madeby the FAO as early as 1974 and published in1977 for 70 countries, covering 90% of the worldspopulation.33,34

The specific indicator, first introduced bySukhatme in 1961,35 is an estimate of the preva-lence of people with insufficient access to food in apopulation. It is based on the concept of adequacy ofDEC relative to dietary energy requirement (DER)and is calculatedusing informationon availability offood for domestic consumption, food energy needsin the population, and inequality in access to food.

In practice, the distribution of food consump-tion in a population is represented by a probabil-ity distribution for the levels of habitual (i.e., an-nual averages) daily DEC of the average individualin the population.k Such distribution is then usedto estimate the extent of food consumption inad-equacy by evaluating the probability mass below acertain threshold, which is termed the prevalence ofundernourishment.36 It can be described as equiv-alent to the probability that a randomly selected

kThe average or representative individual in the popula-tion is a statistical concept that summarizes the distri-bution of the population by age, gender, body mass, andphysical activity levels.



individual in the population is found to be consum-ing, on a regular basis during the year, an amountof food that is insufficient to provide the energyrequired for an active and healthy life.

Validity. On the basis of the short descriptiongiven above, it should be recognized that, contrarytowidespreadmisconceptions, the FAOundernour-ishment indicator is not simply an assessment of av-erage availability of food against requirements at thenational level. Rather, it is a carefully devised statis-tical method to estimate the prevalence of dietaryenergy inadequacy in the population on the basisof a valid concept of food deprivation.37 It cannotbe deemed an indicator of household food security,however, because it is intended to be applied to theentire population and not to individual households,even if data collected from households are of cru-cial importance to inform the measure. Since thePOU is composed of information regarding foodsupply as well as access, it can provide informationregarding the extent to which high prevalence ratesof undernourishment are due to low levels of overallfood availability as opposed to particularly unequaldistribution of access to food.

Reliability. Accuracy of POU estimates has beenthe subject of a long debate.3841 The main issuewas the choice of the threshold used to evaluatethe probability of DEC inadequacy. Critics ques-tioned the use of a single threshold, claiming thatit would imply large biases in the estimate.40,41 Itturns out that the arguments advanced in support-ing this claim are based on the wrong presumptionthat estimates are obtained from the analysis of themarginal distributionofDECas derived froma jointbivariate distribution, requiring observation of bothDEC and DER. This criticism has been shown to beunfounded.38,42

Even if there is no convincing argument to claimthat POU estimates may be systematically biased,questions canbe raisedon their precision. FAObasestheir estimates on three major sources of data: (1)official data on population size and composition;(2) production and trade data from official sources;and (3) data on food acquisition from nationallyrepresentative Household Income and ExpenditureSurveys (HIES). Indirect methods are used whendata are insufficient to provide direct estimates ofthe needed parameters, as it has often been the case

for the parameters describing the inequality in foodconsumption.37

Errors in the estimates may be due to the qual-ity of the data used to inform the estimates of theparameters of the model and to the methods andassumptions made to impute them indirectly.

Population data are used both to calculate foodavailability per capita and to estimate the range ofDER necessary to assess the threshold used for clas-sification. Through their impact on the calculatedaverage dietary energy supply (DES), periodic re-visions of official population assessments have thusdirect consequences on the POU.

Production and trade data are input into the com-pilation of food balance sheets (FBSs), which pro-vide the estimate of total availability of food in acountry.42 The net supply of each food source isconverted into dietary energy, and an estimate ofthe per capita DES is obtained by dividing the totalDES by the countrys population size.43

FBSs are routinelyproducedby theFAOfor a largenumber of countries. Concerns have been raisedwith respect to the reliability of the underlying of-ficial data on production and trade, and of the ac-curacy of assessment of nonfood uses, especially ofstock variations. Such concerns are particularly se-rious when referred to the individual componentsof the elements that compose an FBS, or to theoverall estimate in any single year, which may beaffected by measurement errors that are difficultto quantify. Efforts are made to reduce the poten-tial impact of such errors on the POU by cross-validating the various elements of the balance, forexample, by checking the consistency of feed useof crop products against assessments of the sizeof livestock herds in the country, or by reportingestimates as 3-year averages, thus greatly reducingthe impact of inaccurate measures of yearly stockvariations.

Another concern regarding the use of FBSs toestimate average DEC relates to the difficulty in ac-counting for food losses. Food supply, as recordedin the FBS, may overestimate the quantity availablefor household consumption when food losses at thelevel of retail distribution are not negligible, as re-cently suggested.44

Feasibility. The parametric approach informingundernourishment estimates is at the root of its fea-sibility. Series of POU estimates for most countries



in the world have been published yearly by the FAOsince 1999. This has been possible only through useof a series of indirectmethods to estimate theneededparameters in the absence of direct evidence, whichmay prove to be quite unreliable. When new evi-dence has become available, such as, for example,when suitable data from surveys have been madeavailable to revise the indirectly estimated coeffi-cient of variations, the series of POU have beensignificantly revised, for example, with the releaseof the 2012 edition of the State of Food Insecurityin the World.

Survey-based indicatorsSpurredby the concerns on the reliability of POUes-timates and by the rising popularity of using house-hold surveys to inform poverty analyses,4547 foodsecurity assessments have been increasingly basedon data from household surveys.19,48,49

In principle, if household surveys collected in-formation on relevant concepts of food consump-tion and on the characteristics needed to assess therequirements of household members, it would bepossible to identify households that do not con-sume enough food and to measure the extent offood inadequacy at the household level. Which dataon household food consumption to collect and howto determine adequacy, however, have proven to bedifficult questions to answer.Anobvious temptationhas been to apply the same concept of dietary energyinadequacy that informs the estimates of the POUat the national level in the analysis of survey data toproduce individual- or household-level measures offood consumption inadequacy.

Measures of DEC can be obtained by convertingthe quantities of food items reported as acquiredor consumed by a household or an individual intothe corresponding dietary energy. Accumulatedevidence suggests, however, that these are notreliable measures of the actual dietary energyintakes of individuals. A recent experiment with theLiving Standard Measurement Survey, for example,demonstrates how different designs of the foodconsumption module can lead to differences ofup to 884 kcal even in the average DEC measuredover comparable groups of households in arepresentative survey in Tanzania.50

Much work thus still needs to be done to im-prove quantitative food consumption data collec-tion through large-scale representative surveys in

order to inform reliablemeasures of household foodconsumption adequacy.18,51 Even if systematic mea-surement errors can successfully be reduced, quanti-tative food data collection will remain an expensiveendeavor. For this reason, researchers and agenciesover the past two decades have attempted to de-fine indicators that could be informed by easier-to-collect data in the hope that it may still providerelevant information on the adequacy of food con-sumption.

Dietary diversity scoreDietary diversity has been considered one approachto assess adequacy of food consumption at thehousehold level for use as a food security indicator.Quantitative indicators of dietary diversity, termeddietary diversity scores (DDSs), have been definedas simple counts of the number of food groups con-sumed in a reference period.52 (For a discussionof other possible dietary diversity measures, see,among others, Drescher et al.53).

Validity. There is no doubt that variety is a keyelement of high-quality diets and that variety ordiversity can be meaningfully measured. The rel-evant issue in this discussion though is whether,and to what extent, variations in the DDS reflectdifferences in the food security status of an indi-vidual or a household and do so in a consistentway.

The HDDS was developed by the Food and Nu-trition Technical Assistance (FANTA) Project as aproxy measure of household food access,54 whereasindividual dietary diversity measures reflect nutri-ent adequacy of the diet, as validated against quan-titative 24-h intake nutrient data. Both are definedas how many of a predefined set of different foodgroups are consumed by an individual or a house-hold over a given reference period, even though thenumber and composition of the food groups mayvary.55

The original validation study of household di-etary diversity as a food security indicator lead-ing to development of the HDDSwas a 10-country analysis that showed a strong associationbetween the number of food groups consumed bythe household during the previous 24 h and mea-sures of per capita consumption and dietary energyavailability, as obtained from data collected on thesame households.56 The HDDS tool could thus be agood, and much less expensive, substitute for more



demanding consumption modules to assess foodaccess problems in a group of households.

A subsequent validation of the HDDS was in-cluded in a three-country analysis comparing useof the FCS and different versions of DDSs, includ-ing the HDDS.57 Both the FCS and the variousDDSs were found to be significantly, although notstrongly, correlated with measures of calorie con-sumption as gauged from 7-day household recalldata.

An alternative approach has been to validate DDSas proxies for the food access dimension of foodsecurity conceptualized more broadly than simpledietary energy adequacy. A number of field stud-ies have found that household dietary diversity isconsistently associated with household food secu-rity, as measured through the Household Food In-security Access Scale (HFIAS), with food expendi-ture, and with various indicators of socioeconomicstatus.5861

Cross-analysis of the HDDS with householdfood security status can indeed help to understandthe impact of food insecurity on access to specificfoods, thus guiding policy and programming.In a two-round survey across seasons in CentralMozambique, food-insecure households had lowerfish consumption than food-secure households atboth pre- and postharvest times, and had a 66%reduction in fish consumption in the postharvesttime compared to a 36% reduction in food-securehouseholds after severe flooding destroyed fishponds.62

Reliability. Even thoughDDSvalues are expressedas integer numbers defined over a finite range, it isstill unclear whether they comprise even an ordi-nal scale of the construct they are meant to capture.Lacking a formal theory that links the number offood groups consumed to levels of either nutrientadequacy or food insecurity, it is difficult to assesshow DDS values obtained in different contexts canbe accurate and precise indicators of the constructthey are intended to capture. There are open ques-tions remaining on the optimal number of foodgroups to consider or the minimal size of servingsneeded for consideration in a food group. No solidconclusions can be thus drawn, in general, on thereliability of DDSs as food security measures. TheHDDS has been validated against household caloricavailability, but no cut-off point has been defined to

classify households with low or adequate householddiversity.56

One of the major attractions of dietary diversitytools is that they can be easily computed with dataobtained through relatively inexpensive collectionmethods as rapid, user-friendly and easily admin-istered low-cost assessment tools.56 Ease of compu-tation, however, has a potential cost in that errorsfor example, in wrongly classifying food items intofood groupsmay induce discrete jumps in the di-versity score. Reliability of the classification can beimproved when food groups are adapted to locallyavailable foods.56

A fair conclusion is that, although they cannot beconsidered comprehensive measures of food secu-rity, DDSs somehow reflect energy consumption atthe household level. Analyzed together with otherfood securityrelated information, they can providea holistic picture of the food security status and itsimpact on access to a diverse diet.

Food consumption scoreThe FCS,which has been used by theWFP to informComprehensive Food Security Vulnerability Assess-ments (CFSVA) and Emergency Food Security As-sessments (EFSA), is defined as the frequency-weighted diet diversity score calculated using thefrequency of consumption of different food groupsconsumed by a household during the 7 days beforethe survey.62

It is computed from data on occurrence and fre-quency of consumption of food from different foodgroups, collected through surveys, and on its valueused to classify households food consumption lev-els as poor, borderline, and acceptable on the basisof the FCS.

Collecting simply the occurrence (yes/no) and thefrequency of consumption (how many days duringthe last week) of a number of food groups was orig-inally thought to be a relatively inexpensive methodto obtain information on the nutritional adequacyof food consumption, in an effort to substitutemoredemanding food intake modules.63

Validity. The FCS is an ambitious attempt atdefining an original concept of food consump-tion that would reflect both quantity and quality,on the basis of a parsimonious set of data col-lected through household surveys. The score aims atsummarizing the three aspects reflected in the food



frequency data: dietary diversity, frequency of foodgroup consumption, and nutritional value of food.

In principle, data reduction techniques, suchas Exploratory Factor Analysis or Latent TraitAnalysis,64 could have been used to identify an un-derlying latent trait captured by the food consump-tion variables, in the hope that such a latent traitcould be interpreted as the appropriate food con-sumption construct. Use of such techniques mighthave helped in defining ways to combine the ob-served elementary variables into a single index orscore.

Multivariate data analysis techniques have beenconsidered in the construction and interpretationof the FCS, and the Food Consumption Analysisguidelines published by the WFP still list principalcomponent analysis (PCA) of the unweighted 7-dayrecall food consumption data and cluster analysis ofthe identified components as very important steps indefining the appropriate classifications. It remainsunclear though if and how results of these analyseshave been used to define the weighting scheme.65

The adopted solution to compute a weighted sumof the truncated frequency of consumption of eightfood groups, with weights based on what has beendescribed as an interpretation by a team of analystsof nutrient density, which is a term used to sub-jectively describe a food groups quality in terms ofcaloric density, macro and micro nutrient content,and actual quantities typically eaten,58,66 appearsto be rather ad hoc.

Subsequent validation studies of the FCS58,67 andpublished reportsl do not help in resolving the am-biguity regarding which construct the FCS has beendesigned to capture.

Validation has been attempted in two ways: bymeasuring the correlation of household-level FCSswith measures of household dietary energy avail-ability and by comparing classifications of house-holds on the basis of the FCS with those of thesame households obtained by using dietary energyinadequacy. As for the first, FCSs revealed correla-tions with calorie availability in line with those ofvarious definitions of the DDS, whichas already

l A query of WFPs Survey Data Portal (http://nada.vam.wfp.org/index.php/catalog) involving a search forFood Consumption Score in the variable descriptionsreturned 24 surveys (query run on September 14, 2014).

commentedare statistically significant, but not ofa magnitude sufficient to claim that the FCS wouldaccurately predict household average DEC, a resultconfirmed by other studies.68 It has been suggestedthat improvements in FCS ability to predict calorieconsumption could be obtained by excluding foodgroups consumed in small quantities,58 but the rec-ommendationdoesnot appear tohavebeen adoptedthus far in refining the FCSmethodology, as it seemsto be difficult in practice to exclude small quantities.

Regarding the second type of validation, cross-tabulation of food consumption groups based onthe FCS with those based on calorie consumptionsuffer from the drawbacks that are (1) conditionalon acceptance of the appropriateness of the thresh-olds used to classify caloric consumption, and (2)highly dependent on the particular dataset beingused.

Similar exercises have been used to fine-tune thethresholds used for classification with the FCS forspecific datasets. No thresholds could be found thatwould yield classifications close to those obtainedwith caloric consumption thresholds, even for dif-ferent areas in a country.69

All of these attempts have been, thus far, dis-appointing. It has not yet been possible to iden-tify suitable thresholds, leading to the conclusionthat the current FCS needs to be improved andthat, in any case, it will be necessary to go be-yond the score to provide inputs into nutrition-sensitive programming.69 This raises serious con-cerns regarding the possibility of creating a globallyvalid standard classification on the basis of the FCS.

Reliability. According to the WFP, the FCS hasbeen a reliable indicator of food insecurity in allthe CFSVAs and EFSAswhere it has been applied.69

No information is provided, however, to understandhow reliability is assessed. As is the case for the sim-pler HDDS, it is not clear that the FCS compriseseven a simple ordinal scale of food consumption ad-equacy over the numeric range where it is defined.Measures of accuracy or precision would requirethat the actual scale of adequacy be defined in a waythat would allow for an assessment of the likely ex-tent and direction of errors inmeasuring the under-lying construct, something that is simply impossiblegiven the current definition of the indicator and theexisting evidence.



The FCS lacks minimal characteristics, startingfrom a proper definition of the construct it aimsat measuring, to assess precision and reliability ofthemeasures and to ensure comparability across ap-plications. These properties are considered crucialfor a measure of food security to be adopted in aninternational monitoring framework.

Experience-based food insecurity scalesThe problems with the analysis of food consump-tiondata, and theneed formeasures encompassing abroader concept of food insecurity, created the con-text within which experience-based food insecurityscales emerged.

Ethnographic research carried out in the UnitedStates to understand the experience of food inse-curity and hunger among low-income householdsrevealed it to be a process initially characterizedby uncertainty and anxiety, associated with worryabout being able to get enough food. As condi-tions worsened, it resulted in a decreased amountof stored food in the home, followed by worsen-ing quality and variety of the diet. With increas-ing severity of the food insecurity condition, thequantity of food consumed per meal was decreased,portion sizes were reduced, and households wereforced to skip meals. In the more severe situations,adults and children experienced hunger and did noteat for a whole day or more. Different experientialdomainsuncertainty and worry about food, con-sumption of low-quality food or unbalanced diets,and reductions in the quantity of food consumedwere identified as defining characteristics of house-hold food insecurity.7072

This conceptual understanding of the experienceof food insecurity formed the basis to develop theU.S. Household Food Security SurveyModule (U.S.HFSSM), which has been applied annually in theUnited States to monitor the food security situa-tion since 1995,73 and has served as a model for thecreation ofmany other household experience-basedfood insecurity scales around theworld,7477 includ-ing for the Food Insecurity Experience Scale (FIES)recently developed at the FAO.78

Validity. The substantial innovation broughtabout by experience-based food insecuritymeasuresis the possibility of capturing the food insecurity sit-uation directly and analyzing it from a behavioralperspective, on the basis of the well-grounded as-sumptions that (1) food insecurity is a condition

that may affect overall welfare in different ways, de-pending on the type and strength of the constraintsfaced; (2) food insecurity is characterized by behav-ioral responses to the different conditions, and (3)both the conditions that characterize it and the over-all welfare impact of food insecurity can be locatedalong a scale of severity. This makes it possible tospeak legitimately of subjects not only with respectto being food insecure or not, but also as beingmoreor less food insecure than others, depending on thesituations they report to be experiencing.

From an analytic point of view, the approach isfoundedon conceptualization of the severity of foodinsecurity as a latent attribute that, even thoughinherently unobservable, can be measured over aone-dimensional scale.

Dealing with latent trait measurement signifi-cantly changes the perspective of how to look atvalidity (and reliability) of measurement tools.79

In this context, a precise causal direction runningfrom the concept to the evidence, from the mea-surand to the data, defines validity; namely, thatchanges in the latent trait induce detectable changesin the data used for measurement.80 When ap-plied to experience-based food security scales, thisis equivalent to requiring that, for example, a higherseverity of food insecurity increases the probabil-ity of occurrence of experiences in those domains,which will therefore be more frequently observed inrepresentative samples of the population.

Validity of a particular experience-based scale asa food security measurement tool thus depends ontwo conditions: (1) that the severity of food inse-curity indeed involves the domains that have beenconsidered in creating the items that compose thescales, and (2) that the occurrence of experiences inthose domains can be reliably detected and mean-ingfully linked, albeit in a probabilistic sense, to theseverity of food insecurity.

The existence of fundamental domains of foodinsecurity experiences as identified by the pio-neering ethnographic research in the United Stateshas been extensively validated. Additional ethno-graphic studies aimed at understanding the experi-ence of hunger from the perspective of the elderly,and of immigrants in the United States and low-income families inQuebec,Canada, revealed similarpatterns.8185 Although the original ethnographicstudy was based on a small number of householdsin a wealthy country, a review conducted years later



of experience-based scales in over 20 countries andsettings around the world concluded that these di-mensions of the experience of hunger appear tobe common across cultures.78 These analyses sug-gest that these common domains are robust acrosscultures, in their broad definition and ranking ofseverity.78,86,87

Verification of the second requirement, namelyto theoretically and empirically validate the linkbetween observed experiences and a measure offood insecurity severity, is possible through appli-cation of the Rasch model,88 a probabilistic mea-surement method pioneered in psychometrics andeducational testing that bears fundamental similar-ities with latent variable analysis and discrete choicemodels in econometrics.8991 As applied to food in-security, the Rasch model assumes that the proba-bility that a respondent characterized by a certainlevel of food insecurity would report having ex-perienced the condition described by one of theitems can be represented by an increasing func-tion of the distance between the respondents po-sition on the scale and that of the item.92 Themodel further assumes that items are ranked in thesame order of severity by all respondents indepen-dently of their food insecurity condition, and thatthe more food insecure a respondent is, the morelikely it is that he or she would affirm any givenitem.

This rather simple logic defines the criteriathroughwhich the relative severity both of the itemsand of the respondents can be estimated and, mostimportantly, allows evaluating the empirical valid-ity of any given set of items as contributing to themeasure of a single, underlying one-dimensionaltrait. This is obtained by evaluating the degree ofconsistency between observed patterns of responsesand those that would be expected if the restrictionsimposed by the theory were true.

The logic behind the analytic procedure used toreach a conclusion on the validity of the measuresis as follows: first, as the probability of affirming anitem is expected to decrease with the severity of theexperience it represents, the order of severity of thevarious items included in the scale is derived fromthe shares of respondents that affirm them.m Item

m This makes sense, because, unless there is a problemin understanding the questions, the fewer the number

Response Theory (IRT) in general and the Raschmodel in particular provide formal tests of adher-ence of the data to the theoretical requirements forinvariant measurement through the analysis of itemmisfit statistics.93 Measures of item infit statisticsand of reliability can be used to ascertain whetherall of the items included in the scale, or at leasta subset of them, conform to the requirements ofthe theory. Virtually all studies reporting applica-tions of experience on the basis of food insecuritymeasures in a broad range of countries and condi-tions include an analysis of infit statistics confirm-ing, in the vast majority of the cases, adequacy ofthe item chosen.9396,n In a recent application ofthe eight-item FIES questionnaire in 20 countriesacross Latin America, Africa, and Asia, only oneof the items in just one instance revealed an infitvalue of 1.4, slightly higher than ideal, but still pro-ductive for measurement. All other infits were inthe range 0.841.24, indicating minimal violationof the assumptions that the Rasch model imposes,and confirming that the current version of the FIESis indeed adequate for internationally comparableuse.97

In conclusion, we can say that there are strongconceptual and empirical bases to claim that thefundamental construct underlying these tools, bestdescribed as resource-constraint food insecurity98

is indeed a valid and measurable concept.

Reliability. It has been a rather standard prac-tice to test accuracy of the experience-based scalesby studying associations between the measures ob-tainedwith the scale and the classifications they pro-duce with those obtained using other variables thatare theoretically part of the same construct food in-security and/or that would vary in an expected wayacross different levels of food insecurity, when thesevariables are measured concurrently in the samesubjects.

of people report having suffered from an experience, themore severe (on average) that experience must be.nOf the various different categories of items that had beentested for use in food security scales, only those relatedto the experience of having to procure food in sociallyunacceptable ways were found to be inadequate, as theywould not meet the required standards by showing infitvalues outside the range 0.81.2. See Coates et al.77



National studies conducted in Brazil,99,100

Mexico,101 Bolivia, Burkina Faso, and thePhilippines,102 showed an inverse relationship be-tween the severity of household food insecurityand the consumption or purchase of high nutri-tional quality foods. A study in Albania showedthat dietary diversity (number of food groups con-sumed in a 24-h period) in three vulnerable ar-eas declined with increasing severity of householdfood insecurity, consistent with the underlying the-oretical construct of food insecurity.103 Similar re-sults had been previously found in rural Mexico.104

Explorations have been made of associations be-tween biological indicators of nutritional statusand experience-based measures of food insecurity.One study conducted with children in Colombiareported increasing probability of being stuntedwith increasing severity of food insecurity, as mea-sured by an experience-based scale.105,o Interest-ingly, food insecurity classifications obtained withexperience-based food security scales have showncorrelations with overweight and obesity, especiallyamong women in the United States106 but alsoin some developing countries, such as Brazil andMexico.107,108

While these associations may be informative forprogrammatic purposes, examples of concurrentvalidation cannot be deemed sufficient evidence ofvalidity or reliability of a food security measure-ment tool, as we have argued earlier. Food con-sumption and nutritional status are not prototypegold standards for food security. The experienceof food insecurity, even in the absence of foodenergy deficits or negative effects on nutritionalstatus, is still a serious problem in itself, indicat-ing a violation of the Human Right to AdequateFoodandwithpotentially negative consequences forwell-being.79

Reliability must thus be assessed in a differentway, and IRT provides the means to do so. We havelinked the validity of experience-based food secu-rity scales to the empirical consistency between thedata and the latent trait beingmeasured. Notice that

oResults of such comparisons are likely to be misleadingand difficult to interpret, particularly when food insecu-rity refers to the condition of adults, or when the timing ofthe assessments of food insecurity and nutritional statusis not properly considered.

this does not mean, however, that each and everyset of responses to an experience-based question-naire must be fully consistent, since reliability canbe assessed as a measure of the degree of empir-ical consistency. Discrepancies in actual responsesin relation to the pattern expected according to theunderlying theory (i.e., a respondent who affirmsa given item also affirms all less severe items, andupon denying a given item, also denies all the moresevere ones) may exist without undermining valid-ity, as they can be attributed either to normal sam-ple variability or to measurement error. IRTmodelsmake it possible to assess the size and distributionof such discrepancies to draw conclusions on thereliability of the measures obtained. This is one ofthe aspects that sets these methods apart from theother methods to measure food security that havebeen presented in this paper. This has very relevantimplications for the possibility of defining a properglobal reference standard for food securitymeasure-ment and the comparability of measures obtainedin different countries and different years. Analyz-ing patterns of responses through the lenses of thetheory can guide the choice of which items to in-clude in a global reference scale and which ones toconsider common when reporting measures in onecountry or in 1 year, as compared to such commonstandards.

This is still a relatively unexplored feature of thesemethods but results thus far are extremely encour-aging. Beginning in 2014, the FAO has inserted theFIES into the Gallup RWorld Poll (GWP), a branchof Gallup, Inc. conducting nationally representa-tive surveys annually in more than 150 countries.Preliminary analysis of the results obtained from20 countries revealed measures of Rasch reliabilityranging from 0.69 to 0.78, with a median of 0.73(Rasch is a measure of overall fit of the data to themeasurement model, with theoretical values rang-ing from 0 to 1, with 1 indicating a perfect fit). Thisis a remarkable result, given that the countries cov-ered, including Russia, Brazil, Indonesia, and SierraLeone, among others, are very different from eachother with respect to food insecurity prevalence.109

The same data have led the FAO to define a pro-visional version of a global food insecurity severityreference scale on the basis of the FIES that canbe used for equating prevalence of food security atmedium and high levels of severity across coun-tries and years. Research is also ongoing to develop



methods to equate prevalence levels obtainedthrough other experience-based food security scalesused in many countries, such as national scales de-veloped in Brazil and Mexico, and the ELCSA andtheHFIASused inmany countries in LatinAmerica,Asia, and Africa.

Conclusions and RecommendationsAn overview of the evolution of the internationaldebate regarding the concept of food security hasrevealed how the problems of understanding, mon-itoring, and communicating about it have beendeeply interlinked.

The measurement issue has received much atten-tion and many indicators have been proposed andapplied over the years in recognition of the com-plexity of food security as defined by the interna-tional community. But, as we have argued, this hashappened often without due consideration of thetheoretical requirements to define proper measures.

Even though the daily practice of fighting foodinsecurity does not allow for exhaustive and long-term inquiries on validity and reliability, whichmaybe practical only within the most rigorous aca-demic environments, a solid understanding by thoseproposing the different indicators of the importanceof adherence to minimal quality criteria for mea-surement is imperative.

To shed light on the issue, we have discussed afew of the methods proposed to compile indica-tors of food security and have critically evaluatedthe available evidence regarding their validity andreliability. Many proxy indicators of food insecu-rity have been based on food consumption data.However, feasibility of collecting sufficiently pre-cise data to allow reliable assessment on a regularbasis, and with enough disaggregation of the datato be useful in providing meaningful policy guid-ance, is compromised by the burden it may imposeon the resources available to countries for such anendeavor.

The need for cost effectiveness and timeliness ofreporting has driven the search for innovative waysto obtain reliable measures with universal applica-bility. For food consumption, attempts have beenmade at defining relatively simple indicatorstheHDDS and the FCSto be applied at the householdlevel. These are based on concepts of food adequacythat explicitly include dietary quality considerationsas captured by dietary diversity, and they have the

advantage of being relatively parsimonious with re-gard to data collection. Unfortunately, the lack ofan explicit model linking dietary diversity and foodconsumption frequency to food consumption ad-equacy and food security has prevented, thus far,establishing their validity as worldwide comparablemeasures of food access at the household or indi-vidual levels.

Improvements in measures of food consumptionadequacy are certainly necessary and worthwhile.However, from the perspective of assessing food in-security, they will always leave something to be de-sired. These types of measures are consistent withan approach that relies on proxy measures. An ana-lytic framework to link different aspects of the foodconsumption experiencesuch as the quantity andquality of food consumed, or the extent and fre-quency of episodes of inadequacyis necessary tointerpret them as relevant for the overall food secu-rity condition of an individual or household.

The use of experienced-based food insecurityscales, from this perspective, has indeed been aparadigmatic change, both conceptually and inpractice. It has provided a clear unifying conceptleading to the development of direct measures offood insecurity severity at the household and indi-vidual levels. It puts peoples experiences and behav-ioral responses at the core of the definition of whatfood insecurity means, promoting the shift towardthird generational indicators that had been in-voked long ago.110,111 Recent experience shows notonly that these tools produce valid, cross-countrycomparable measures, but also that they can be ap-plied in a relatively simple and inexpensive way.

Indicators of the severity of experienced food in-security canbeusedwithinbroader conceptsofmul-tidimensional poverty, consistently with a focus onunequal access to food and sociocultural aspects ofthe experience of hunger,112 and can enhance in-vestigation of associations between food insecurityand various forms of malnutrition, including bothunder- andovernutrition. Food insecurity can affecthealth and well-being in many ways, with poten-tially negative mental and social consequences, inaddition to physical well-being, even in the absenceof measurable negative effects on nutritional sta-tus. One of the unique contributions of experience-based food security measures is that they also cap-ture psychosocial aspects associated with anxiety oruncertainty regarding the ability to procure enough



food, an element of the experience of hunger andfood insecurity that othermeasures do not consider.

It is imperative that those in charge of food se-curity assessments understand in depth the under-lying conceptual framework and therefore the prosand cons of the different indicators. Moreover, theyneed to assure that the framework is understood bythose interpreting the outcomes of the assessmentsand by those designing the food security programsand policies. One of the results of our discussionon validity of current approaches to food securitymeasures is that validating indicators on the basisof their association with other food security mea-sures cannot be deemed conclusive, because it isnot possible to determine which of the comparedmeasures validates the other. Although an adequateassessment of food insecurity need not be restrictedto the indicators described above, it is worrisomethat the measurement field is widely populated byan array of indicators that have not been scrutinizedto the level required by the importance and mag-nitude of the phenomenon that we aim to measureand address.113

In order to promote food security more effec-tively, it is imperative to be able to measure itin a more valid and reliable way. The lack of agold standard should not discourage validation ef-forts, which would greatly contribute to identifyingways of improving the suitability of the proposedtools and to designing programs that better tar-get those in need. Given the very limited resourcesavailable in daily practice, easy-to-apply tools arecertainly in high demand. Nevertheless, this real-ity should not be seen as justification for acquir-ing and applying tools only because of their lowcost.

On the basis of the evidence we have reviewed,we conclude that experience-based food insecuritymeasures are themost promising tools among thosecurrently available and constitute a valuable instru-ment that can be applied together with other indi-cators to better understand the determinants andconsequences of household and individual food in-security. They are foundedonavalid concept of foodinsecurity that covers common domains across cul-tures and socioeconomic conditions and thus havethe potential to form the basis for a valid measureworldwide. Moreover, they promise to effectivelycombine desirable features of cost effectiveness andrapidity of monitoring with sufficient precision and

reliability, and to do so in a way that is applicableworldwide and at the global, national, and subna-tional levels. The work that the FAO is conductingto promote the FIES will go a long way toward pro-viding the necessary evidence to guide successfulpractical implementation of a global measurementstandard. Research is demonstrating that the FIES,even in its current form, is indeed capable of com-parably measuring the prevalence of food insecu-rity at different levels of severity across populationsthat differ in their linguistic, cultural, and socioe-conomic aspects. Results so far are extremely en-couraging and point to the possibility that a globalstandard tomeasure household and individual foodinsecurity is possible. This is a worthy endeavor, andwe can anticipate that the development of such aglobal standard will generate enormous benefits forresearchers, analysts, advocates, and policy makersinterested in eradicating food insecurity and hungerby providing the necessary tool to monitor progressin a timely and consistent way, both within andacross countries.

Conflicts of interestC.C., T.B., and A.K. are affiliated with the FAO,which is involved in food security measurement.

References1. FAO. 1946. World Food Survey. Food and Agriculture Or-

ganization of the United Nations, Washington, DC.2. FAO. 1952. Second World Food Survey. Food and Agricul-

ture Organization of the United Nations, Rome, Italy.3. FAO. 1963. ThirdWorld Food Survey. Freedom fromHunger

Campaign Basic Study No. 11. Food and Agriculture Orga-nization of the United Nations, Rome, Italy.

4. Sen. 1983. Poverty and Famines: An Essay on Entitlementand Deprivation. Oxford University Press.

5. Shaw, D.J. 2007. World Food Security. A History since 1945.Palgrave McMillan.

6. Gibson, M. The Feeding of Nations. Redefining food securityfor the 21st century. Boca Raton, FL: CRC Press.

7. Maxwell, S. 1992. Food security: a post-modern perspec-tive. Food Policy 21: 155170.

8. Hoddinott, J. 1999. Choosing Outcome Indicators of House-hold Food Security. Technical Guide No. 7. Washington, DC:International Food Policy Research Institute.

9. Rome declaration on world food security. http://www.fao.org/docrep/003/w3613e/w3613e00.HTM.

10. U.N. General Assembly. 2001. Road map towards theimplementation of the United Nations MillenniumDeclaration. Report of the Secretary-General. http://millenniumindicators.un.org/unsd/mdg/Resources/Static/Products/SGReports/56_326/a_56_326e.pdf.

11. FAO. 1996. The sixth world food survey.



12. UNICEF. The state of world children. http://www.unicef.org/sowc/.

13. FAO. 2003. Measurement and assessment of food dep-rivation and undernutrition. http://www.fao.org/docrep/005/Y4249E/y4249e00.htm.

14. FAO. 1996. Rome Declaration on World Food Security andWorld Food Summit Plan of Action. World Food Summit1317 November 1996. Rome, Italy.

15. deHaen,H., S. Klasen&M.Qaim.What dowe really know?Metrics for food insecurity and undernutrition. Food Policy36: 760769.

16. Coates, J. 2013. Build it back better: deconstructing food se-curity for improved measurement and action. Global FoodSecurity 2: 188194.

17. Jones, A.D., F.M. Ngure, G. Pelto & S.L. Young. 2013. Whatare we assessing when we measure food security? A com-pendium and review of current metrics. Adv. Nutr. 4: 481505; doi:10.3945/an.113.004119.

18. Carletto, G., A. Zezza & R. Banerjee. 2013. Towards bet-ter measurement of household food security: harmonizingindicators and the role of household surveys. Global FoodSecurity 2: 3040.

19. Perez-Escamilla, R. & A.M. Segall-Correa. 2008. Food in-security measurement and indicators. Rev. Nutr. [online],21(Suppl.): 15s26s.

20. Frongillo, E. 1999.Validationofmeasuresof food insecurityand hunger. J. Nutr. 129: 506S509S.

21. InternationalVocabulary ofMetrologyBasic andGeneralConcepts andAssociatedTerms (VIM), 3rd ed. http://www.bipm.org/utils/common/documents/jcgm/JCGM_200_2012.pdf.

22. http://en.wikipedia.org/wiki/Validity_(statistics). (Italicsadded).

23. Markus, K.A. & D. Borsboom. 2013. Frontiers of Test Va-lidity Theory. Measurement, Causation, andMeaning. Rout-ledgeMultivariate Application Series. Section 2.5. New York:Routledge.

24. American Educational Research Association, AmericanPsychological Association, National Council on Measure-ment in Education. 2009. Standards for Educational AndPsychological Testing. Washington, DC: American Educa-tional Research Association.

25. Cook, D.A. & T.J. Beckman. 2006. Current concepts in va-lidity and reliability for psychometric instruments: theoryand application. Am. J. Med. 119: 166.e7166.e16.

26. Borsboom, D. 2005. Measuring the Mind. Cambridge, UK:Cambridge University Press.

27. International vocabulary ofmetrology (cit.), definition 2.3,p. 17.

28. Gates, B. 2013. Improving the lives of the poor will take asteady, thoughtful revolution in development data. Devel-opment Asia. November.

29. FAO. 2012 The State of Food Insecurity in the World 2012.Economic Growth is Necessary but not Sufficient to AccelerateReduction of Hunger and Malnutrition. Technical Note.http://www.fao.org/fileadmin/templates/es/SOFI_2012/sofi_technical_note.pdf.

30. WHO/FAO/UNU. 2004. Human Energy Requirements. Re-port of a Joint FAO/WHO/UNU Expert Consultation, Rome,Italy, 1724 October 2001. Rome: FAO.

31. WHO/FAO/UNU. 2007. Protein and Amino Acid Re-quirements in Human Nutrition. Report of a jointFAO/WHO/UNU expert consultation (WHO Technical Re-port Series 935). Geneva: WHO.

32. Burlingame, B., C.Nishida, R.Uauy&R.Weisell. 2009. Fatsand fatty acids in human in nutrition. Joint FAO/WHOExpert Consultation. November 1014, 2008, Geneva,Switzerland. Special Issue. Ann. Nutr. Metab. 55: 5300.

33. FAO. 1975. Population, food supply and agricultural de-velopment. In The State of Food and Agriculture 1974.Chap. 3, pp. 93196. See in particular the sections titledDimensions and causes of hunger and malnutrition, pp.103111, and Indicators of the incidence of undernutri-tion, pp. 149151.

34. FAO. 1977. The fourth world food survey. FAO StatisticsSeriesNo. 11andFAOFoodandNutritionSeriesNo. 10. FoodandAgricultureOrganizationof theUnitedNations, Rome,Italy. See inparticular Part II:Dimensions ofmalnutrition.

35. Sukhatme, P.V. 1961. TheWorlds hunger and future needsin food supplies. J. R. Stat. Soc. Ser. A (Gen.) 124: 463525.

36. Naiken. 2003. FAO methodology for estimating the preva-lence of undernourishment. In Proceedings of the In-ternational Scientific Symposium on Measurement andAssessment of Food Deprivation and Undernutrition, 2628 Jun 2002, FAO, Rome, Italy. Agriculture and Eco-nomic Development Analysis Division, pp. 742. http://www.fao.org/docrep/005/Y4249E/y4249e06.htm#bm06.

37. Cafiero, C. What Do We Really Know About Food Secu-rity?NationalBureauofEconomicResearchWorkingPaper18861. http://www.nber.org/papers/w18861.

38. Osmani, S.R. 1992. Nutrition and Poverty. UNU-WIDERStudies in Development Economics. Oxford: ClarendonPress.

39. Svedberg, P. 1999. 841 million undernourished?World De-velop. 27: 20812098.

40. Svedberg, P. 2000. Poverty and Undernutrition. Theory,Measurement and Policy. UNU-WIDER Studies in Devel-opment Economics. Oxford University Press.

41. Naiken, L. 2007. The Probability Distribution Frameworkfor Estimating the Prevalence of Undernourishment:Exploding the Myth of the Bivariate Distribution.FAO Statistics Division Working Paper Series No. ESS/ESSG/009e FAO, Rome, August. http://www.fao.org/fileadmin/templates/ess/documents/food_security_statistics/working_paper_series/WP009e.pdf.

42. http://faostat.fao.org/site/368/default.aspx.43. FAO. 2001. Food Balance Sheets. A Handbook. Rome.

http://www.fao.org/docrep/003/x9892e/x9892e00.htm.Food and Agriculture Organization of the United Nations,Rome, Italy.

44. FAO. 2011. Global Food Losses and Food Waste Extent,Causes andPrevention. Food andAgricultureOrganizationof the United Nations, Rome.

45. Deaton, A. 1997. The Analysis of Household Surveys: A Mi-croeconometric Approach to Development Policy. Baltimore:Johns Hopkins University Press.

46. Haughton, D. & J. Haughton. 2011. Living Standards An-alytics Development through the Lens of Household SurveyData. New York: Springer.



47. Foster, J., S. Set, M. Lokshin & Z. Suraja. 2012. A UnifiedApproach to Measuring Poverty and Inequality. Theory andPractice. Streamlined Analysis with ADePT Software. Wash-ington, DC: World Bank.

48. Smith, L.C., H. Alderman & D. Aduayom. 2006. food in-security in sub-Saharan Africa. New estimates from house-hold expenditure surveys. Research Report 146. InternationalFood Policy Research Institute, Washington, DC.

49. Moltedo, A., N. Troubat, M. Lokshin & Z. Suraja. 2014.Analyzing Food Security Using Household Survey Data:Streamlined Analysis with ADePT Software. Washington,DC: World Bank.

50. De Weerdt, J., K. Beegle, J. Friedman & J. Gibson. 2014.TheChallenge ofMeasuringHunger. PolicyResearchWorkingPaper Series 6736. The World Bank, Washington, DC.

51. Smith, L.C., O Dupriez & N. Troubat. 2013. Assessmentof the reliability and relevance of the food data collectedin national household consumption and expenditure sur-veys. International Household Survey Network, Food andAgriculture Organization of the United Nations andWorldBank. February 2014.

52. Kant, A.K., A. Schatzkin, T.B. Haris, et al. 1993. Dietarydiversity and subsequent mortality in the First NationalHealth and Nutriton Examination Survey EpidemiologicFollow-up Study. Am. J. Clin. Nutr. 57: 434440.

53. Drescher, L.S., S. Thiele & G.B.M. Mensik. 2007. A newindex to measure healthy food diversity better reflects ahealthy diet than traditional measures. J. Nutr. 137: 647651.

54. Swindale, A. & P. Bilinsky. 2006. Household DietaryDiversity Score (HDDS) for Measurement of HouseholdFood Access: Indicator Guide (v.2). Washington, DC: FHI360/FANTA.

55. FAO. 2010. Guidelines for Measuring Household andIndividual Dietary Diversity. Rome. http://www.fao.org/docrep/014/i1983e/i1983e00.pdf.

56. Hoddinott, J. & Y. Yohannes. 2002. Dietary diversity asa household food security indicator. Food and NutritionTechnical Assistance Project, Academy for Educational De-velopment, Washington, DC

57. Wiesmann, D., L. Bassett, T. Benson & J. Hoddinott. 2009.Validation of the World Food Programmes Food Consump-tion Score and Alternative Indicators of Household Food Se-curity. IFPRI Discussion Paper 00870. International FoodPolicy Research Institute (IFPRI), Washington, DC

58. Kennedy, G.,M. Razes&T. Ballard. 2014. Measurement ofdietary diversity for monitoring the impact of food basedapproaches. In Improving Diets and Nutrition: Food BasedApproaches. B. Thompson&L.Amoroso, Eds. Boston ,MA:FAO/CABI.

59. Thorne-Lyman, A.L., N. Valpiani, K. Sun, et al. 2010.Household dietary diversity and food expenditures areclosely linked in rural Bangladesh, increasing the risk ofmalnutrition due to the financial crisis. J. Nutr. 140: 182S188S.

60. Taruvinga, A., V. Muchenje & A. Mushunje. 2013. Deter-minants of rural household dietary diversity: the case ofAmatole and Nyandeni districts, South Africa. Intl. J. De-velop. Sustain. 2: 22332247.

61. FAO. 2008. Report on use of the household food insecu-rity access scale and household dietary diversity scorein two survey rounds in Manica and Sofala Provinces,Mozambique, 20062007. http://www.foodsec.org/fileadmin/user_upload/eufaofsi4dm/docs/moz_diet.pdf.

62. World Food Program-VAM. 2008. Food ConsumptionAnalysis: Calculation and Use of the Food ConsumptionScore in Food Security Analysis. p. 8. http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp197216.pdf.

63. Kennedy, G.L., A. Berardo, C. Papavero, et al. 2010. Proxymeasures of household food consumption for food secu-rity assessment and surveillance: comparison of the house-hold dietary diversity and food consumption scores. PublicHealth Nutr. 13: 20102018.

64. Bartholomew, D.J., F. Steele, I. Moustaki & J. Galbraith.2008. Analysis of Multivariate Social Science Data, 2nd ed.Boca Raton, FL: CRC Press.

65. World Food Program-VAM. 2008. Food ConsumptionAnalysis. Calculation and Use of the Food ConsumptionScore in Food Security Analysis. p. 5.

66. World Food Program-VAM. 2008. Food ConsumptionAnalysis. Calculation and Use of the Food ConsumptionScore inFoodSecurityAnalysis. http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp197216.pdf. p. 19, footnote 14.

67. Tiwari, S., E. Skoufias &M. Sherpa. 2013. Shorter, Cheaper,Quicker, Better: Linking Measures of Household Food Secu-rity to Nutritional Outcomes in Bangladesh, Nepal, Pakistan,Uganda, and Tanzania. Policy Research Working Paper No.WPS 6584. World Bank, Washington, DC.

68. Mathiassen, A., S. Montembault & K. Ogden. 2012.Household Food Consumption: looking beyond the score.Presentation delivered at the International Conference onDietary and Physical Activity Methods (ICDAM), Rome17.5.2012, Session B-6 Measuring Dietary Diversity. http://www.fao.org/fileadmin/templates/nutrition_assessment/Oral_presentations/B6_Mathiassen.pdf.

69. World Food Program-VAM. 2009. Comprehensive FoodSecurity & Vulnerability Analysis Guidelines. 1st ed. http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp203208.pdf, p. 212.

70. Radimer, K., C.M. Olson & C.C. Campbell. 1990. Devel-opment of indicators to assess hunger. J. Nutr. 120(Suppl.):1544S1548S.

71. Radimer, K.L., C.M. Olson, J.C. Green, et al. 1992. Un-derstanding hunger and developing indicators to assess itin women and children. J. Nutr. Educ. 24(Suppl.): 36S44S.

72. Radimer, K.L. 2002. Measurement of household foodsecurity in the USA and other industrialized coun-tries. Public Health Nutr. 5(6A): 859864. DOI:http://dx.doi.org/10.1079/PHN2002385.

73. Hamilton, W., J.T. Cook, W.W. Thompson, et al. 1997.Household food security in the United States in 1995: tech-nical report. Technical Report prepared for theUSDAFoodand Consumer Service, Alexandria, VA.

74. Segall-Correa, A.M., L. Marn-Leon & R. Perez-Escamilla.2010. Escala Brasileira de Medida da Inseguranca



Alimentar e Nutricional EBIA: Validacao, usos e im-portancia para polticas publicas. In Fome Zero. UmaHistoria Brasileira, Vol. 3, pp. 2643. Brasil: Ministerio deDesenvolvimento Social e Combate a` Fome.

75. Deitchler, M., T.J. Ballard, A. Swindale & J. Coates. 2010.Validation of ameasure of household hunger for cross-culturaluse.TechnicalReport.Washington,DC:FANTA-2.Academyfor Educational Development, Washington, DC.

76. Perez-Escamilla, R., H. Melgar-Quinonez, M. Nord, M.C.et al. 2007. Escala Latinoamericana y Caribena de Seguri-dad Alimentaria (ELCSA) [Latinamerican and CaribbeanFood Security Scale]. Perspectivas en Nutricion Humana,Colombia 2007(Suppl.): 117134.

77. Coates, J., E.A. Frongillo, B.L. Rogers, et al. 2006. Com-monalities in the experience of household food insecurityacross cultures: what are measures missing? J. Nutr. 136:1438S.

78. Ballard T.J., A.W. Kepple & C. Cafiero. 2013. The Food In-security Experience Scale. Development of a Global Stan-dard for Monitoring Hunger Worldwide. Rome: Foodand Agriculture Organization of the United Nations.http://www.fao.org/3/a-as583e.pdf.

79. See Borsboom(cit.), Chapter 3, andMarkus andBorsboom(cit.), section 2.5.

80. See Borsboom (cit.), Chapter 6.81. Wolfe, W.S., C.M. Olson, A. Kendall & E.A. Frongillo Jr.

1996. Understanding food insecurity in the elderly: a con-ceptual framework. J. Nutr. Educ. 28: 92100.

82. Wolfe, W.S., E.A. Frongillo & P. Valois. 2003. Understand-ing the experience of food insecurity by elders suggestsways to improve its measurement. J. Nutr. 133: 27622769.

83. Hamelin, A.M., M. Beaudry & J.P. Habicht. 2002. Char-acterization of household food insecurity in Quebec: foodand feelings. Soc. Sci. Med. 54: 119132.

84. KaiserL.L.,H.Melgar-Quinonez,C.Lamp, et al. 2002. Foodsecurity and nutritional outcomes of Latino Preschoolers.J. Am. Diet. Assoc. 102: 924929.

85. Kaiser L.L., H. Melgar-Quinonez, M.S. Townsend, et al.2003. Food insecurity and food supplies in Latino house-holds with young children. J. Nutr. Educ. Behav. 35: 148153.

86. Swindale, A. & P. Bilinsky. 2006. Development of a univer-sally applicable household food insecurity measurementtool: process, current status, andoutstanding issues. J.Nutr.136: 1449S1452S.

87. Melgar-Quinonez, H., M.C. Alvarez Uribe, Z.Y. Fon-seca Centeno, et al. 2010. Psychometric characteristics ofthe Food Security Scale (ELCSA) applied in Colombia,Guatemala y Mexico. Revista de Seguranca Alimentar e Nu-tricional, Campinas 17: 4860.

88. Rasch, G. 1960. Probabilistic Models for Some Intelli-gence and Attainment Tests. Copenhagen: Danish Insti-tute for Educational Research. Reprinted with a fore-word and afterword by B.D. Wright in 1980 by the Uni-versity of Chicago Press, and in 1993 by MESA Press,Chicago, IL.

89. McFadden. 1980. Econometric models for probabilisticchoice among products, J. Business 53: S13S29.

90. Maddala, G.S. 1983. Limited-Dependent and QualitativeVariables in Econometrics. Cambridge, UK: CambridgeUniversity Press (for a comprehensive review).

91. Train, K. 2009. Discrete Choice Methods with Simulation.2nd ed. New York: Cambridge University Press.

92. Nord, M. 2014. Introduction to Item Response Theory ap-plied to Food Security Measurement: Basic Concepts, Pa-rameters, and Statistics. Technical Paper. Rome: FAO.http://www.fao.org/economic/ess/ess-fs/voices/en.

93. Alvarez, M.C., C. Montoya, A. Estrada, H. Melgar-Quinonez. 2006. Validacion de escala de percepcion de laseguridad alimentaria domestica en Antioquia, Colombia.Salud Publica Mex 48: 474481.

94. Melgar-Quinonez, H. & M. Hackett.2008. Measuringhousehold food security: the global experience. Rev. Nutr.21: 27s37s.

95. Nord,M. 2012. Assessing potential technical enhancementsto the u.s. household food security measures. EconomicResearch Service of the United States Department of Agri-culture Technical Bulletin No. (TB-1936), 130 pp.

96. Faye, O, A. Baschieri, J. Falkingham & K. Muindi. 2011.Hunger and food insecurity in Nairobis Slums: an assess-ment using IRTmodels. J. UrbanHealth 88(Suppl. 2): 235255.

97. FAO.2014.Validity of theFood InsecurityExperience Scale.Preliminary results from application through the GallupWorld PollTM. Mimeo. Rome: FAO.

98. Ohls, J., L. Radbill & A. Schirm. 2001. Household foodsecurity in the United States, 19951997: technical issuesand statistical report. Final Report of the Project to Analyze1996 and 1997 Food Security Data. December 2001, p. 6.

99. Segall-Correa, A.M., R. Perez-Escamilla, L. Marn-Leon,et al. 2008. Evaluation of household insecurity in Brazil:validity assessment in diverse socio

validity and reliability of food security measures

Documents