OverviewThe Center for Innovation in Learning and Teaching (CILT) implements an automated lecture recording system at the University of Cape Town (UCT). The current software solutions are proprietary and use expensive Pan-Tilt-Zoom (PTZ) cameras.

Natural Presenter Tracking In 4K Video TRACK4K

The ProblemCILT is experimenting with cheaper 4K (3840x2160 pixels) wide-angle cameras to record lectures. Since the resultant videos are very large (~2GB), we were asked to create an open-source automated lecture recording system capable of 1) extracting a reduced resolution stream from the 4K video input, and 2) ensuring the board content is legible and in the frame at all times. Our solution uses computer vision algorithmsto determine a suitable smaller crop region to extract from each 4K frame.

Board DetectionWe find candidate boards by their edges and save them as rectangles. These rectangles are evaluated to determine which are really boards. The amount of content on a board determines when last it was used. All the boards are then captured in an enclosing frame.

EDGE DETECTION RECTANGLES FEATURES

Board Segment

ENCLOSING RECTANGLE

DETECT MOTION SEGMENT MOTION SELECT LECTURER

Lecturer TrackingWe detect movement by comparing the differences between frames and storing them in rectangles. The lecturer is then found in one of these rectangles based on the time spent on screen.

Virtual Cinematographer

We use lecturer positions to decide whether (and how) to pan the virtual camera such that both lecturer and the board being referred to are kept in-frame.

Lecturer Position

Tim

e

Left Motion

Right Motion

Noise

PAN OPERATIONS PANNING OUTPUT

ConclusionWe detect boards successfully in well-lit conditions. We detect (and track) the lecturer in most cases. We produce output better than the current system can. In conclusion, the project achieved its stated objectives.

Computer Science DepartmentUniversity of Cape TownPrivate Box X3Rondebosch7001

A/Prof. Patrick MaraisMr. Stephen Marquard

SupervisorsCharles FitzhenryBlackboard Segmentation

Maximilian HahnLecturer Tracking

Mohamed Tanweer KhatiebVirtual Cinematographer

www.track4k.co.za