visual geometry group department of engineering science...
TRANSCRIPT
![Page 1: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/1.jpg)
Image Classification
Dr. Ernesto CotoResearch Software Engineer
Visual Geometry GroupDepartment of Engineering Science, University of Oxford
![Page 2: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/2.jpg)
Outline
2
BBC News Search
– Image Classification Engine
–Demo
Sharing our work
–VGG's Image Classification (VIC) engine
–Demo
– Features
–Availability
–Deployment
–Data Ingestion
![Page 3: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/3.jpg)
BBC News Search
Powerful public web engine for visual search overthe BBC News dataset ( ~17K programmes, ~10Khours of video, ~5M keyframes, ~1.5 TB)
3Available at: http://zeus.robots.ox.ac.uk/bbc_search/
![Page 4: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/4.jpg)
BBC News Search
Objects/Scenes (Image Classification): Searches of abroad nature: e.g. animals (horses, etc.), vehicles (cars, etc.),styles (gothic architecture, etc.)
Instances (Image Matching): Searches for a specificobject or building (e.g Big Ben)
Text: Searches for instances of a text string
People: Searches of people or face types (e.g. Tony Blair)
4
![Page 5: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/5.jpg)
BBC News Search
The Image Classification engine can be input text. The textis used to downloaded training images from a searchengine. This starts a search for similar images in the dataset
Matching images in the dataset are ranked and displayed
If the input is an image, the image is used to train themodel
5
![Page 6: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/6.jpg)
Sharing our work: VIC
You can use the same Image Classification engine forsearching on your own dataset !
VGG's Image Classification (VIC) engine
6
![Page 7: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/7.jpg)
Sample dataset: COCO2014
Common Object in Context (COCO) – 2014 version
Complex everyday scenes containing common objects intheir natural context.
Includes 80 object categories
More than 300K images
– We use a subset that includes objects and text (~14Kimages)
http://mscoco.org/7
![Page 8: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/8.jpg)
VIC: Features
Query by entering a text or an image
Training images automatically downloaded fromGoogle
Automatic training, classification and result ranking
User management
Query refining (Uber classifiers) and caching
Curated queries
Data ingestion:
– Use your own dataset and/or extend it later
– Define your own metadata
8
![Page 9: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/9.jpg)
VIC: Availability
Supported platforms: Linux and Mac. Currentlyworking on the Windows deployment
Deployment options:
–Simple deployment in your computer via Dockertechnology
– For advanced users, native deployment from thesource code is also possible
9Images take from: https://www.docker.com
![Page 10: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/10.jpg)
VIC: Simple Deployment
1) Install Docker (needs privileged access)
2) Install the VIC application (installer files provided)
3) Instruct VIC to ingest your data
10
111
2
3
![Page 11: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/11.jpg)
VIC: Mac Deployment Demo
Install Docker for Mac
Install VIC using provided installer
BEFORE starting VIC, copy your own data to VIC folder
Start the VIC application and perform the data Ingestion
11
Admin tools with different options to add/remove data
from the Image Classification engine
![Page 12: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/12.jpg)
VIC: Advanced Deployment
Native deployment successfully tested on Ubuntu 14 LTS,macOs Sierra. Installation scripts can be provided.
Frontend consists of a Django Application and the imagedownloading engine (Google web engine only)
– Could provide support for other downloading engines
– Source code: https://gitlab.com/vgg/vgg_frontend
Backend consists of a python wrapper and a native imageclassification component
– Source code: https://gitlab.com/vgg/vgg_classifier
Backend and Frontend communicate via an API. Newbackends can be incorporated.
Docker deployment requires Docker v17.03.1-ce.12
![Page 13: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/13.jpg)
VIC: Stats
Data ingestion:
– Dataset: ~14K images, ~27 min (multi-threaded)
– Negatives: ~17K images, ~1 hour (single-thread)
Backend data footprint
– Dataset: ~2.4 GB
– Extracted dataset features: 7.6 MB
– Negative images: ~416.7 MB
– Extracted negative features: 9.5 MB
Backend memory footprint: 185 MB (reducible with PQ)
One cached query: 18 MB (16 MB of training images)
Docker image: 1.4 GB13
![Page 14: Visual Geometry Group Department of Engineering Science ...vgg/projects/seebibyte/events/...–Simple deployment in your computer via Docker technology –For advanced users, native](https://reader035.vdocument.in/reader035/viewer/2022071219/6056c39fa470941fb427e296/html5/thumbnails/14.jpg)
Image Classification
Dr. Ernesto CotoResearch Software Engineer
Visual Geometry GroupDepartment of Engineering Science, University of Oxford