artificial intelligence in plant phenomics · artificial intelligence applications to agriculture...
TRANSCRIPT
![Page 1: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/1.jpg)
Artificial Intelligence in Plant Phenomics
Ian StavnessAssociate ProfessorComputer ScienceUniversity of Saskatchewan
Artificial Intelligence Applications to AgricultureTexas A&M16 July 2020
![Page 2: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/2.jpg)
Crop Development Center, Saskatoon, Canada
![Page 3: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/3.jpg)
Roots/SoilCrop imaging Deep learning Plant breeding
![Page 4: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/4.jpg)
Selecting features is hard
Sadeghi-Tehran et al. (2017). Automated Method to Determine Two Critical Growth Stages of Wheat: Heading and Flowering. Front. in Plant Sci., 8(February), 1–14.
![Page 5: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/5.jpg)
Deep learning is representation learning
Sadeghi-Tehran et al. (2017). Automated Method to Determine Two Critical Growth Stages of Wheat: Heading and Flowering. Front. in Plant Sci., 8(February), 1–14.
![Page 6: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/6.jpg)
Imaging & Deep Learning for Agriculture
Seed scale Global scale
crop damage, crop insurance
crop health,precision management
From RGB to NIR: Predicting of near infrared reflectance from visible spectrum aerial images of crops
Masi Aslahi, Kevin Stanley, Ian Stavness
Abstract Regular crop monitoring is essential for crop quality and health. However, the traditional crop monitoring methods are labor-intensive and error prone. Recently, different imagery techniques are becoming increasingly viable in precise farming. For example, a drone may be flying over the field once a week to capture images with a multi-spectral camera which is able to capture near-infrared wavelengths (NIR) for further data analysis. Unfortunately, human experts must still view the resulting images to determine the crop condition. Also, flying a drone equipped with a multi-spectral cam-era is still costly. A method which could reliably speed up the data analysis, and which is cost effective is highly favorable. Here, we de-scribe the use of a generative adversarial network (GAN) [1] on drone images as a non-destructive method to transform RBG to NIR. The GAN model offers a practical and theoretically sound method of mapping RGB to NIR, and generates images comparable to the ground truth.
Computer Science
Data acquisition
Method
Fig 3: Lentil trial—different growth stages
Future Work The proposed method is handy and cost-effective and will reduce human intervention, and consequently enhance the yield. Also, this approach is robust against the low quality images and environmental noise. Future works will consider other crops that have completely different plant morphology.
Reference
Fig 4: Normalized difference vegetation index (NDVI)
Fig 1: Data acquisition process in the wheat/lentil breeding field located at Kernen near Saskatoon
Fig 2: Training procedure to map RGB aerial images to NIR
Results RGB Actual NIR Fake NIR
Earl
y st
age
Mid
sta
geLa
te s
tage
Normalized difference vegetation index (NDVI) [2] plays an important role in predicting agricultural production. To deter-mine the mass of greenness in an area, visible and non-visible lights are collected.
Fig 5: Biomass of actual and fake lentil trial image
[1] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio , Generative Adversarial Network . Advances in neural information processing systems, 2014 , 2672-2680 .
[2] https://en.wikipedia.org/wiki/Normalized_difference_vegetation_index
Plant scale Field scaleidentifying plants,estimating traits
early disease detection
yield prediction,price forecasting
NASA
weather prediction,logistics
seed phenotyping,provenance
automated seed inspection
![Page 7: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/7.jpg)
Learned Features across ScalesSpatial Spectral Temporal
(Mishra et al. 2017)
NASA
“Camera On A Stick”
![Page 8: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/8.jpg)
Image-based Phenotyping
Object Detection
Classification/ Regression
Semantic Segmentation
InstanceSegmentation
Local Regression
Deep Plant Phenomicshttps://github.com/p2irc/DeepPlantPhenomics
![Page 9: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/9.jpg)
Challenge #1: Large diverse datasets
???
![Page 10: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/10.jpg)
New open datasets
https://global-wheat.com
Global Wheat Head Dataset
![Page 11: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/11.jpg)
Global Wheat Head Detection Dataset
David, E., Madec, S., Sadeghi-Tehran, P., Aasen, H., Zheng, B., Liu, S., Pozniak, C., Stavness, I., Guo, W. (2020). Global Wheat Head Detection (GWHD) dataset. Plant Phenomics, in press.
![Page 12: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/12.jpg)
Global Wheat Head Detection Dataset
![Page 13: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/13.jpg)
Competition for CVPPP 2020
![Page 14: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/14.jpg)
Global Wheat Data: Future Contributions
https://global-wheat.com
![Page 15: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/15.jpg)
PlotVision
![Page 16: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/16.jpg)
PlotVision
![Page 17: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/17.jpg)
Challenge #2: Image annotation
Box aroundeach object
Label forwhole image
Draw outlines Draw outlinesfor each object
Dot on eachobject
Low cost High cost
![Page 18: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/18.jpg)
Latent Space PhenotypingPhenotype-to-genotyping mapping for stress resistance
Ubbens, J., Cieslak, M., Prusinkiewicz, P., Parkin, I., Ebersbach, J., & Stavness, I. (2020). Latent space phenotyping: automatic image-based phenotyping for treatment studies. Plant Phenomics, 2020, 5801869
https://github.com/p2irc/LSPlab
![Page 19: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/19.jpg)
Image-based Phenotyping
Images NumericPhenotype GWAS
Tedious annotations!
![Page 20: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/20.jpg)
Latent Space Phenotyping
Images GWAS
![Page 21: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/21.jpg)
Latent variable models
Encoder Decoder
LatentRepresentation
Orig
inal
Imag
e
Reco
nstru
cted
Im
age
![Page 22: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/22.jpg)
Latent variable models
Encoder
LatentRepresentation
Orig
inal
Imag
e
Output: Treatment Label
Phenotype forStress Response
![Page 23: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/23.jpg)
Datasets: Setaria RIL
Feldman et al. (2018). Components of Water Use Efficiency Have Unique Genetic Signatures in the Model C 4 Grass Setaria. Plant Phys., 178(2), 699–715.
![Page 24: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/24.jpg)
Datasets: Canola NAM
![Page 25: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/25.jpg)
Datasets: Synthetic Arabidopsis
• Genomic data from the A. thaliana polymorphism database• Images generated from a 3D L-system model
![Page 26: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/26.jpg)
Embedding Process
Feature Extractor Classifier
![Page 27: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/27.jpg)
Decoding Process
![Page 28: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/28.jpg)
Example decoded images
![Page 29: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/29.jpg)
Measuring Response-to-Treatment
![Page 30: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/30.jpg)
treatedcontrol
Trait Value
Results: Synthetic Arabidopsis
-log 10
(P v
alue
)
Chromosome
-log 10
(P v
alue
)
Chromosome
treatedcontrol
Trait Value
What if we use simple image distance instead?
![Page 31: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/31.jpg)
Results: Setaria
treatedcontrol
Trait Value
0
1
2
3
4
Chromosome
LOD
1 2 3 4 5 6 7 8 9
[email protected]@33.5
[email protected]@33.5
![Page 32: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/32.jpg)
Results: Canola
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
50
100
150
200
250
pre-treatment during treatment post-treatment
Experiment Phase
Trai
t Val
ue
Early Flowering – ControlEarly Flowering – TreatedIntermed. Flowering – Control
Late Flowering – ControlLate Flowering – Treated
Intermed. Flowering – Treated
●
●
●
●
●
●
● ●
●
● ●
●
●
●
●
●
●
50
100
150
200
250
pre-treatment during treatment post-treatment
Experiment Phase
Trai
t Val
ue
Early Flowering – ControlEarly Flowering – TreatedIntermed. Flowering – Control
Late Flowering – ControlLate Flowering – Treated
Intermed. Flowering – Treated
![Page 33: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/33.jpg)
Limitation: Explainability
![Page 34: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/34.jpg)
AutoCount: Unsupervised Organ Counting
Ubbens, J., Ayalew, T., Shirtliffe, S., Josuttes, A., Pozniak, C. & Stavness, I. (2020). AutoCount: Unsupervised Segmentation and Counting of Organs in Field Images. ECCV Workshops, 2020, to appear.
To appear: www.plant-phenotyping.org/CVPPP2020
![Page 35: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/35.jpg)
AutoCount: Unsupervised Organ Counting
Ubbens, J., Ayalew, T., Shirtliffe, S., Josuttes, A., Pozniak, C. & Stavness, I. (2020). AutoCount: Unsupervised Segmentation and Counting of Organs in Field Images. ECCV Workshops, 2020, to appear.
To appear: www.plant-phenotyping.org/CVPPP2020
![Page 36: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/36.jpg)
AutoCount: Unsupervised Organ Counting
Ubbens, J., Ayalew, T., Shirtliffe, S., Josuttes, A., Pozniak, C. & Stavness, I. (2020). AutoCount: Unsupervised Segmentation and Counting of Organs in Field Images. ECCV Workshops, 2020, to appear.
To appear: www.plant-phenotyping.org/CVPPP2020
![Page 37: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/37.jpg)
Domain Adaptation for Organ CountingTo appear: www.plant-phenotyping.org/CVPPP2020
Source:Indoor labeled dataset
Target:Outdoor Unlabeled dataset
GWHD
CropQuant
Ubbens, J., Ayalew, T., & Stavness, I. (2020). Unsupervised Domain Adaptation For Plant Organ Counting. ECCV Workshops, 2020, to appear.
![Page 38: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/38.jpg)
Domain Adaptation for Organ Counting
Ubbens, J., Ayalew, T., & Stavness, I. (2020). Unsupervised Domain Adaptation For Plant Organ Counting. ECCV Workshops, 2020, to appear.
To appear: www.plant-phenotyping.org/CVPPP2020
![Page 39: Artificial Intelligence in Plant Phenomics · Artificial Intelligence Applications to Agriculture Texas A&M 16 July 2020. Crop Development Center, Saskatoon, Canada. ... identifying](https://reader034.vdocument.in/reader034/viewer/2022043006/5f8ec3c89a5a6172ad108ed1/html5/thumbnails/39.jpg)
Domain Adaptation for Organ CountingTo appear: www.plant-phenotyping.org/CVPPP2020
Ubbens, J., Ayalew, T., & Stavness, I. (2020). Unsupervised Domain Adaptation For Plant Organ Counting. ECCV Workshops, 2020, to appear.