introducing indigo ag crop indices (in chi & ndvi) a new ...€¦ · contribute value at many...

50 South B.B. King Blvd | Memphis, TN 38103 | (844) 828-0240 | [email protected] | www.IndigoAg.com

Vegetation indices (VIs) are used in many life science applications, including agriculture. Many of the mainstream VIs in use today were developed years ago as generalist plant health indicators that could be used in any vegetated land cover. We propose a new framework for building VIs that are specifically optimized for agriculture; we call these Crop Indices (CIs). The Indigo Ag suite of CIs, including our own version of common metrics like NDVI and EVI as well as proprietary metric called the Indigo Ag Crop Health Index (IN CHI), represent a novel set of tools for assisting with both in-season and inter-annual decision support. In tailoring IN CHI specifically for agricultural monitoring, we observed a much wider dynamic range across the full spectrum of end of season yield relative to other mainstream indices that are prone to saturating at the upper end of their ranges. Additionally, by using high quality crop masks to focus our attention on where crops are growing, we improve our CI correlations with end of season outcomes by as much as 33%. As part of our earth monitoring system, Indigo Ag generates global CIs daily, summarized across multiple geographies. In introducing this new product, we propose that CIs (Including our own NDVI) have the potential to make significant improvements over traditional VIs and will contribute value at many scales - field, regional, national, and global. We see a dramatic range of industry applications, including index-based insurance (IBI), farm management, real estate, commodities trading, supply chain risk and food security.

Vegetation indices (VIs) developed from optical remote sensing satellites are commonly used to measure and monitor plant health (see background material). VIs rely on the physics of photosynthesizing plants; healthy, green leaves absorb and reflect specific wavelengths of light (Figure 1), and remote sensing scientists are able to observe these relationships from space. Since the 1970s, VIs have been designed to maximize the correlation with plant health by using mathematical combinations of image spectral channels (for example, NDVI: Rouse et al. 1974), and thanks to satellites, we have global, high cadence coverage with a long historical archive and consistent calibration.

Introducing Indigo Ag Crop Indices (IN CHI & NDVI)

Abstract

Background

Problem Statement: Optimizing VIs for Agriculture

A new suite of signals for grains and oilseeds

15 March 2018


VIs have been used for many life science applications, including agriculture. Their potential has been noted by many commercial actors and they are considered standard for market intelligence products (e.g. Bloomberg, Reuters) and they have made an impact at a variety of scales. However, they were largely developed to reflect plant health in general - from grasslands to shrublands to forests - and have not been adequately tailored to agriculture

We propose ‘Crop Index’ (CI) as a more suitable term for a specific type of vegetation index that has been optimized for use in agricultural decision making. A crop index refers to a product or suite of products that are tightly coupled to the geography and phenology associated with particular crops or crop types.

CIs have the potential to make significant improvements over traditional VIs for a number of players in the Ag space. By having Agriculture-specific indices, we can contribute value at many scales - field, regional, national, and global. We can help farmers, governments, investors, and underwriters understand, in-season, how this year is progressing versus other years, which will allow them to make inferences about end of season production as well as serve as another tool in their decision support systems.

Figure 1. In this figure, the ‘red edge’ is clearly evident as a sharp spike in reflectance on the boundary between visible (blue, green, red) and near infrared light. Source: UN Office for Outer Space Affairs

Motivation

Our Solution

Why CIs vs. VIs? Making a Case


We propose that Indigo Ag is positioned to add value in this area because our CIs specifically 1) utilize our domain knowledge of where the crops are, 2) are focused formulae - both existing and homegrown - that are intended specifically assessing crop health, as opposed to the health of all types of green vegetation on earth, and 3) respect that crops are on a clock that doesn’t necessarily follow the date on the calendar. By incorporating weather and seed technology, we are able to track crops in different regions based on the plant’s own cycle and specific to the conditions experienced by the plant in-season.

A well-functioning CI should provide a better basis for making decisions related to the crop in question. Based on feedback from both agronomy and finance professionals, our conclusion is that any candidate CI should excel in the following dimensions:

Most of our testing of individual crop index approaches to date has focused on accuracy and consistency, including some testing of complementarity. Simplicity is a constant challenge, as there is often tension between it and achieving the absolute highest accuracy. Quality decisions are made at the platform level -- latency, frequency, and reliability are attributes of the entire Argus-Kernel system, and individual components, such as a CI, must meet rigorous specifications to be included in our technology stack.

Accuracy. The index should be reliably correlated with outcomes that matter for the crop. In many cases, end of season yield is the most available yardstick.

Consistency. The index should correlate to outcomes consistently through time and across geographies. In temporal space, stability is important seasonally as well as over a decadal time span in order to capture the rapid pace of change in agricultural technologies. Given the scope of the global balance sheet, the index should also perform well across regions and countries.

Complementarity. Given the utility of weather indexes, any viable CI should also prove itself complimentary to these and other long standing measurement tools.

Simplicity. As is the case with weather, any strong CI should be simple enough to explain, to allow for reproducibility and clarity.

Quality. And finally, as is the case with modern weather services, a useful CI must be produced with the lowest latency, highest frequency, and highest reliability possible.

Why CIs vs. VIs? Making a Case - Continued

How do we Know when we’re Successful?

What makes this hard?


In our experience, several challenges are associated with building useful CIs that support agricultural decision-making:

Our process for delivering Indigo Ag indices, including our own proprietary Indigo Ag Crop Health Index (IN CHI), is outlined in Figure 2. We begin with a highly processed set of satellite images, measurements that have undergone atmospheric correction, cloud screening, look angle adjustments, and a host of quality assurance protocols (steps 1 - 2). We then calculate TL-CIs and extract distinct values for specific crops (steps 3 - 4). The crop-specific masks that make this process possible are updated annually, resulting in yearly TL-CI offerings based on revised views of crop locations. Finally, Indigo Ag delivers those crop-specific index values at county, state, and national levels (steps 5 - 6).

Over the next several sections, we will share some early evidence of the impact of these key factors, and where we are in the journey with our Indigo Ag indexes.

Reflectance (remote sensing). The remote sensing is not straightforward. Polar orbiting satellites, look angles, atmospheric correction, clouds, decay of sensors and calibration, and multi-sensor needs are all potential impediments. This challenge is true for VIs as well.

Crop maps (masking). Vegetation indices are often developed and monitored without respect to distinct geographies. Knowing where the crops are matters, both at regional and finer scales.

Index design (formula). The formula for the index itself matters. Common VIs were not specifically designed for crops; we can further tune the algorithms to make them more useful for agricultural applications.

Crop phenology (timing). The growing season is not uniform through time. When making comparisons over wide areas, plant timing is a confounding factor; we are exploring techniques to identify and weight the right moments.

Technology. Quality, near-real-time information is paramount to good decision-making. CIs must be developed on a platform that fights latency, maximizes frequency, minimizes errors, and achieves the best overall availability possible.

What makes this hard? - Continued

IN Crop Index Processing Overview

1.

2.

3.

4.

5.


IN Crop Index Processing Overview - Continued

Figure 2. The six key steps in creating TL CHI.

There are hundreds of satellites in orbit, and on the back of the most important scientific satellites are hundreds of products and offerings. A thorough understanding of remote sensing is required to choose the right sensors and the right products. Each decision point is crucial in producing quality agricultural observations at enough locations, often enough, over a long enough period of time to provide meaningful insights

We chose MODIS NBAR surface reflectance products for Version 1.0 of our suite of IN Crop Indices. NASA’s Version 6 Nadir Bidirectional reflectance distribution function Adjusted Reflectance (NBAR) product is a daily 16-day composite of acquisitions from both the Terra and Aqua sensors at 500 m spatial resolution. Within each 16-day window, the best representative pixel is chosen from the entire pool of available observations and corrected to simulate collection from a nadir view, removing the effects of view angle. MODIS NBAR images are also standardized to remove atmospheric effects, providing a more stable and consistent product useful for time series analyses. We utilize the MODIS NBAR historical archive back to 2003, when both Terra and Aqua were first online together.

As part of our earth monitoring system, Indigo Ag generates a suite of global CIs on a daily basis, summarized across multiple geographies. These indices include common VIs, like NDVI, EVI2, and NDWI, that are specifically pointed at agricultural areas of interest (see Crop Masking section), as well as our own proprietary metrics. Given our choice of satellite inputs (see Section 1, Reflectance), we have a complete phenological record dating back to 2003 (Figure 3) for each of our crop indexes. This long, consistent historical time series allows us to quickly detect and report anomalies during the growing season for any of our crops of interest.

Reflectance

Index design, production, and performance


Additionally, Indigo Ag has designed a novel Crop Health Index (IN CHI) using a ratio of bands that best captures changes in crop health that are linked to end of season outcomes (Figure 4). The design process was informed by well-established metrics such as the Normalized Difference Vegetation Index (NDVI) and Enhanced Vegetation Index (EVI) that capitalize on the physical properties of photosynthesizing plants; however, our focus was specifically on monitoring agriculture, as opposed to more generalized vegetation indices, which allowed us to tailor the way spectral information was combined in the index.

IN CHI shows promising dynamic range in response to yield. The Indigo Ag Crop Health Index exhibits a much less saturated response to end of season yield relative to many mainstream vegetation indices, in particular versus the most popular VI today, NDVI (Figure 5). With IN CHI, we are able to detect varying index signal even at the top of the yield range; whereas, NDVI, for example, reaches its maximum well before peak yield.

We further observed that the entire suite of IN Crop Indices is useful at different points in the season (Figure 6). Given the design of the individual metrics (i.e., some are more efficient at detecting greenness, others plant water content, for example), it is not surprising that the CIs, though correlated, each provide useful signal with regard to monitoring plant health and progress, as well as to end of season outcomes.

Index design, production, and performance - Continued

Figure 3. IN CHI records extend back to 2003, with a new value generated each day. Here, we show the time series for IN CHI, masked for soybeans, at the US National level.


Figure 4 - The top panel shows a wall-to-wall map of the Indigo Ag Crop Health Index over parts of South America as displayed in our customer-facing product, Kernel. The bottom image of the same area is masked by croplands (see section 3), where the brightest areas are agriculture and darker, grayed out areas are non-agriculture. These bright pixels are the ones we carry through into our zonal summaries, as they are pure crop pixels that provide the cleanest signals for monitoring and modeling. We produce annual masks for all croplands generally as well as for specific crops.


Figure 5. Mainstream VIs (panels B-D) tend to saturate at the top of their ranges when compared against end of season yield. The IN Crop Health Index (panel A) was specifically designed to provide enhanced dynamic range for agricultural applications.

Figure 6. Indigo Ag Crop Index Pearson Correlations to end of season yield. Orange background shading indicates mean national greenup period, green indicates average maturity, and yellow is senescence (Negative values are outside of growing period).


A crucial aspect of developing crop indices is knowing where the crops are, which changes annually in many places, and focusing only on those pixels for crop health monitoring. In the US, we are fortunate to have the Cropland Data Layer (CDL), a crop-specific 30 m spatial resolution product released annually by the USDA National Agricultural Statistics Service (USDA 2017). Figure 7 shows an example of annual crop rotations in several fields in northeastern Nebraska. This type of rotation is typical across the US and demonstrates the importance of having annual crop masks that reflect the variable conditions on the ground each year.

We associate this 30 m CDL product with our 500 m VI pixels by counting the number of crop pixels within each MODIS pixel and calculating the percent cover for each crop, resulting in a continuous fractional cover layer for each crop of interest that is exactly aligned with our CI images. We can then choose a threshold or use the fractional cover as a weight when summarizing CIs within administrative units.

In countries where high resolution CDL-like annual crop maps are not available, producing a fractional cover layer is much more challenging. In these situations we use the Collection 6 annual MODIS land cover product in conjunction with government reported statistics on area harvested (Figure 4, bottom panel).

We have found that masking for specific crops greatly improves Pearson correlations to end-of-season outcomes during the growing season (Figure 8, Table 1). Masking increases the efficacy for every VI at or near peak growing season, when the crop health signal is strongest and correlation to yield is highest (Table 1). We observed correlation improvements of 14% - 33% with masking (depending on time of year).

Crop Masking

Figure 7. The 2017 Cropland Data Layer summarized within field boundaries for a sample area in northeastern Nebraska. Fields outlined in red show callouts that list the crop type in each field since 2012, demonstrating typical crop rotation in the US. The main crops in this image are corn (yellow), soy (green), and alfalfa (pink).


Figure 8. Average daily Pearson correlation coefficients for TL CHI vs. end of season yield for 2003 - 2016 in the US. Orange background shading indicates mean national greenup period, green indicates average maturity, and yellow is senescence.

Table 1. Masked and unmasked Pearson correlation coefficients (r) for Indigo Ag suite of CIs vs. end of season yield on typical peak days during the US growing season. These days of year correspond to the green shaded area in Figure 4

IN CHI NDVIDoY Masked Unmasked Masked Unmasked200 0.580 0.517 0.595 0.446208 0.665 0.580 0.625 0.470216 0.713 0.617 0.642 0.488224 0.712 0.624 0.639 0.498


With this publication, we are unveiling version 1.0 of the Indigo Ag Crop Indices. However, we see this line of research as an ongoing opportunity for improvement. Future advancements include incorporating new satellite and ground data streams, as well as more explicitly integrating phenology into our products.

Future versions of Indigo Ag crop indices will incorporate data from multiple optical sensors, including the additions of Landsat and Sentinel-2. We are also experimenting with incorporating radar signals to better capture the structure and moisture profiles of the vegetation. We recognize that every instrument is a compromise, in many cases between spatial and temporal resolution. Detailed insights into complex processes can be improved with multi-sensor data fusion to maximize cadence, coverage, and spatial refinement. Challenges include cross calibration of multiple instruments and ensuring similar levels of processing/quality (e.g., atmospheric correction, radiometric calibration, etc.) between sensors.

Additionally, we have already begun work to incorporate phenology into our indices. The growing season is not uniform through space and time; when making comparisons over large geographic areas, plant timing is always a confounding factor. In an effort to identify and properly weight the right moments in the season, we are working on using a phenological clock, rather than the calendar date, to measure and compare crop progress both in-season and between seasons in a standardized way. In Figure 9, the top set of images (set A) shows each of our CIs across 2011 - 2016, using calendar date as the clock (including the edges of the season where much of the US is covered in snow, resulting in some extreme variations on the tails). In set B, we see a much clearer view of yearly CI curve morphology when we use a phenological clock. We observe, for instance, that 2012 has a noticeably flatter, wider shape, indicating lower peak CI values than normal. Not coincidentally, 2012 was an historic drought year with extremely low corn yields. Since calendar date does not translate to the same point in the season for all crops in all locations, using a phenological clock is an effective way to normalize for season completeness.

The Future of Indigo Ag Crop Indices


Figure 9. Set A shows seasonal CI progress curves for US corn using Day of Year as the “clock.” Set B shows the same data, but standardized by percent of the season complete. All CI curves are normalized by the average of their peak annual values.


Rouse Jr, J., Haas, R. H., Schell, J. A., & Deering, D. W. (1974). Monitoring vegetation systems in the Great Plains with ERTS.

USDA National Agricultural Statistics Service Cropland Data Layer. 2008 - 2017. Published crop-specific data layer [Online]. Available at https://nassgeodata.gmu.edu/CropScape/ (accessed 03/12/2017). USDA-NASS, Washington, DC.

Introductory topics: - NASA feature on measuring vegetation - ’Red edge’ explained - UN article on vegetation indices

Background on important indices for Indigo Ag: - Normalized Difference Vegetation Index (NDVI) - Enhanced Vegetation Index (EVI) and 2-band EVI (EVI2) - Normalized Difference Wetness Index (NDWI)

More information on Remote Sensing indices and instruments: - NASA webinar on fundamentals of Remote Sensing - NASA ARSET Curricula on remote sensing

The Future of Indigo Ag Crop Indices

Further reading

introducing indigo ag crop indices (in chi & ndvi) a new ...€¦ · contribute value at many...

Documents