data set and ground truth
DESCRIPTION
Data Set and Ground Truth. Data set and Ground truth. Data set contents Data set resources Ground truth contents Ground truth resources. Data set contents. Goal: Capture realistic video sequences with graduate difficulties: lighting variations, occlusions … - PowerPoint PPT PresentationTRANSCRIPT
INRIA, NICE, December 7th-8th 2006
Data set and Ground truth
• Data set contents
• Data set resources
• Ground truth contents
• Ground truth resources
INRIA, NICE, December 7th-8th 2006
Data set contents
Goal:
Capture realistic video sequences with graduate difficulties:
• lighting variations,
• occlusions …
covering predefined scenarios.
INRIA, NICE, December 7th-8th 2006
Data set contents
Video providers, main characteristics
• Inrets: Building Entrance, Indoor/Outdoor, Multi-view
• CEA: Corridor + Road, Indoor/Outdoor, Visible/IR
• RATP: Metro, Indoor, Crowded environment
• iLids: Road, Outdoor, Real street context
• Silogic: Aircraft parking zone, Multi-view
INRIA, NICE, December 7th-8th 2006
Data set contents
5 Topics
• Apron
• Building Corridor
• Building Entrance
• Metro
• Road
INRIA, NICE, December 7th-8th 2006
Data set contents
APRON2 points of 2 points of
viewview
Different Different weather weather
conditionsconditions
INRIA, NICE, December 7th-8th 2006
Data set contents
BUILDING CORRIDOR
Visible and IR clipVisible and IR clip
INRIA, NICE, December 7th-8th 2006
Data set contents
BUILDING ENTRANCE
Entrance andEntrance andCar ParkCar Park 4 points of 4 points of
viewview
INRIA, NICE, December 7th-8th 2006
Data set contents
ROAD
Visible and IR clipVisible and IR clip
Real sceneReal scene
INRIA, NICE, December 7th-8th 2006
Data set and Ground truth
• Data set contents
• Data set resources
• Ground truth contents
• Ground truth resources
INRIA, NICE, December 7th-8th 2006
Data set resources
Data set 1 2 Total
Nb sequences
5 AP
3 BC
19 = 3 BE
4 MO
4 RD
5 AP
3 BC
21 = 3 BE
5 MO
5 RD
10 AP
6 BC
40 = 6 BE
9 MO
9 RD
Nb clips
(x Nb camera)
14 AP
6 BC
41 = 9 BE
4 MO
8 RD
10 AP
6 BC
37 = 9 BE
5 MO
7 RD
24 AP
12 BC
78 = 18 BE
9 MO
15 RD
INRIA, NICE, December 7th-8th 2006
Data set resources
Data set 1 2 Total
Total sequences Duration
38 min 03 s12 min 04s AP
06 min 40s BC
02 min 24s BE
11 min 52s MO
05 min 03s RD
36 min 58 s06 min 33s AP
07 min 24s BC
02 min 46s BE
10 min 16s MO
09 min 59s RD
1 h 15 min 01 s18 min 37s AP
14 min 04s BC
05 min 10s BE
22 min 08s MO
15 min 02s RD
Total Duration 1 h 23 min 19 s 1 h 00 min 02 s 2 h 23 min 21 s
Mean Duration / sequence
1 min 56 s02 min 25s AP
02 min 13s BC
00 min 48s BE
02 min 58s MO
01 min 16s RD
1 min 45 s01 min 19s AP
02 min 28s BC
00 min 55s BE
02 min 03s MO
02 min 00s RD
1 min 50 s
INRIA, NICE, December 7th-8th 2006
Data set resources
Data set 1Participation
(13 participants)2
Participation
(16 participants)
Priority sequences
1 AP
0 BC
1 BE
1 MO
1 RD
92 %
---
69 %
69 %
100 %
1 AP
0 BC
1 BE
1 MO
2 RD
94 %
---
94 %
75 %
94 %
Wished sequences
1 AP
1 BC
1 BE
2 MO
0 RD
46 %
15 %
8 %
0 %
---
1 AP
1 BC
1 BE
2 MO
1 RD
25 %
25 %
19 %
22 %
19 %
Others
AP
BC
BE
MO
RD
25 %
0 %
7 %
0 %
5 %
AP
BC
BE
MO
RD
23 %
15 %
25 %
22 %
19 %
INRIA, NICE, December 7th-8th 2006
Data set resources
• 2 video data set (test + evaluation)
• 2*2 DVD provided
• Documentation (metrics, annotation rules, structures and formats)
• Context and calibration information
• Data base (xls file)
INRIA, NICE, December 7th-8th 2006
Data set resources
Topics Event of interest
APVehicle movements:
empty_area, inside_zone, enters_zone, stopped
BCPerson and objects interactions:
opens/closes (door), picks_up/puts_down, abandoned_baggage, changes_zone, running
BEPerson, vehicle and objects interactions (complex scene):
opens/closes (door), gets_in/gets_out, go_down/up_stairs, door_control, inside_zone, enters/exist_zone, changes_zone
MOAbandoned baggage in crowded environment:
Waiting, picks_up/puts_down, abandoned_baggage, people counting, exchange_objects, (ticket machine)
RDPerson, vehicle and objects interactions (complex scene):
opens/closes (door), gets_(in/on)/gets_(out/of), biking, crossing, walking_with, overtaking
INRIA, NICE, December 7th-8th 2006
Data set and Ground truth
• Data set contents
• Data set resources
• Ground truth contents
• Ground truth resources
INRIA, NICE, December 7th-8th 2006
Ground truth contents
• Context information
• Physical objects of interest and events annotation
Annotation rules provided to participant in a document and updated during the project
Examples…
INRIA, NICE, December 7th-8th 2006
Data set and Ground truth
• Data set contents
• Data set resources
• Ground truth contents
• Ground truth resources
INRIA, NICE, December 7th-8th 2006
Ground truth resources
Data set 1 2 Total
Nb ground truth files
14 AP
3 BC
34 = 9 BE
4 MO
4 RD
10 AP
3 BC
32 = 9 BE
5 MO
5 RD
24 AP
6 BC
66 = 18 BE
9 MO
9 RD
Sequence time1 h 11 min 36 s
(61 470 frames)
48 min 31 s
(56 422 frames)
2 h 00 min 07s
(117 892 frames)
INRIA, NICE, December 7th-8th 2006
Conclusion & Feedback
Difficulties encountered:
• Definition of event
• Annotation subjectivity
Proposition for other evaluation project:
• Ground truth done before data set distribution: Easier to identify difficulties that may encountered participants and add explanations and specific annotation rules
• Ground truth given between the deadline submission and the results processing to be able to correct ground truth before evaluation
-> Time consuming