language and images
DESCRIPTION
Language and images. Abhijit Sarkar. Adjective. Noun. Verbs. Adverbs. Sentence source - UIUC Sentence dataset Comparison using Gist : 4X4 window Test picture source – VOC2012 No object detectors . Using only same category. Rank 2. River Water Red Banking maneuvers. Low Plane Day - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/1.jpg)
Language and images
Abhijit Sarkar
![Page 2: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/2.jpg)
Noun
Verbs
Adjective
Adverbs
![Page 3: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/3.jpg)
• Sentence source - UIUC Sentence dataset
• Comparison using Gist : 4X4 window
• Test picture source – VOC2012
• No object detectors
![Page 4: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/4.jpg)
Using only same category
![Page 5: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/5.jpg)
![Page 6: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/6.jpg)
![Page 7: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/7.jpg)
![Page 8: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/8.jpg)
Rank 2
RiverWaterRedBankingmaneuvers
LowPlaneDaygray
![Page 9: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/9.jpg)
When testing with other category
![Page 10: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/10.jpg)
Rank 1
![Page 11: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/11.jpg)
Rank 2
![Page 12: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/12.jpg)
Rank 2
![Page 13: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/13.jpg)
• Test with any picture
• Take top 10 ranked results (10 out of 1000)
• Try to identify the best
![Page 14: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/14.jpg)
![Page 15: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/15.jpg)
![Page 16: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/16.jpg)
![Page 17: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/17.jpg)
![Page 18: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/18.jpg)
![Page 19: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/19.jpg)
![Page 20: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/20.jpg)
![Page 21: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/21.jpg)
Not so Good results
![Page 22: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/22.jpg)
![Page 23: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/23.jpg)
![Page 24: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/24.jpg)
![Page 25: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/25.jpg)
![Page 26: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/26.jpg)
![Page 27: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/27.jpg)
And some Really Bad …….
![Page 28: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/28.jpg)
![Page 29: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/29.jpg)
![Page 30: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/30.jpg)
![Page 31: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/31.jpg)
Conclusion• Good Results - – includes sky or similar ambience– Airplane – Horse – Bike– Indoor scene– Sofa
• Bad Results– Cat– Dog– Birds
![Page 32: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/32.jpg)
Gender
![Page 33: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/33.jpg)
GenderAge
Nationality
![Page 34: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/34.jpg)
DirectionLeft/ RightFront/ Side
![Page 35: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/35.jpg)
DetailsDirection
![Page 36: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/36.jpg)
Minute Details
![Page 37: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/37.jpg)
Conclusion
• “Sentences are richer than lists of words, because they describe activities, properties of objects”
• We use words like “peaceful”, “Calm”, “Turbulent”, “sweet”, “lone”
![Page 38: Language and images](https://reader036.vdocument.in/reader036/viewer/2022062323/56815ed1550346895dcd61d8/html5/thumbnails/38.jpg)
Thanks•Acknowlegements
–Dr. Devi Parikh
image Credit : Jamie Shotton