Opencast and Sakai at the University of Cape Town

Stephen Marquard, Centre for Innovation in Learning and Teaching, UCTstephen.marquard@uct.ac.za | twitter: stephenmarquard

… and … Get the closeup! Lecturesight and Track4K

||

Opencast at UCT

UCT is a medium-sized research-intensive residential university with about 25,000 students

2016 statistics:

71 venues equipped for automated capture (Galicaster 2)

400 lectures recorded each week

3000 different recordings played back each week

Used by up to 4500 students each week

Unique users per 24 hours (green) and 7 days (blue), Feb 2016 – Feb 2017

Unique recordings viewed over 24 hours (green) and 7 days (blue), Feb 2016 – Feb 2017

|| 3

Sakai and Opencast

Goals: Seamless experience for students and staff – make it easyto manage and access lecture recordings and user-generated video.

Enhanced LTI player: students (responsive, filtering) Enhanced LTI player: site owners (manage recordings, upload) Sakai User and Role Provider (Opencast 2.4) State of the code Future work

Open source projects we use and contribute to:Sakai, Opencast, Galicaster, PyCA, Lecturesight, Track4K

2017/02/28

|| 2023-05-02 4

|| 2023-05-02 5

|| 2023-05-02 6

|| 7

The Sakai User and Role ProviderGoal: make Opencast aware of Sakai users and sites, without duplicating all Sakai users and sites inside Opencast. Delegate Opencast authorization to Sakai.

When a user logs into Opencast, the Sakai User Provider looks up the user in Sakai and gives the Opencast user a set of roles from their Sakai site membership.

The Sakai Role Provider makes Sakai-based ACLs available in Opencast.

Sakai site + role = Opencast role (siteid_role)

Sakai user = Opencast role (ROLE_USER_eid)

You can add these ACLs to a Series or Event in the Opencast UI or through REST endpoints.Therefore you can make an Opencast Series visible to one or more Sakai sites and/or users.

2017/02/28

|| 82017/03/01

Sakai sites and users are verified by calling the REST /direct/site and /direct/user endpoints. If the site and/or users exist, they can be added to Opencast ACLs.

|| 92017/03/01

Users can exist in Opencast only, in Sakai only, or in both Opencast and Sakai.

The Sakai user provider gives the user the ROLE_GROUP_SAKAI role (adds to the Sakai group), and adds roles from the user’s Sakai site membership.

|| 10

Opencast and Sakai: Code

LTI support: included in Opencast since 1.x (but will not save users, so limited to playback and cannot schedule or start workflows - MH-8955).

Sakai User Provider: MH-10871 for Opencast 2.4 (could be back-ported to earlier versions) – makes Sakai users valid in Opencast, populated with roles from Sakai site membership

Ability to add Sakai site roles in the 2.x Admin UI ACLs: MH-12016 for Opencast 2.4

Enhanced LTI tool including filtering and scheduling: available from UCT (https://bitbucket.org/cilt/opencast) with transitional UI

2017/02/28

|| 11

Sakai and Opencast: What’s next

Fully self-service setup for a new series in a Sakai site (add tool and get started)

Easy permission and visibility changes on series and recordings (ACL updates)

Ability to embed an Opencast video in Lessons or other Sakai contexts

Simple Upload workflow for DIY lecture recording and user-generated content (make Opencast the default way to host video in Sakai)

Sakai authorization provider (?): login to Opencast with Sakai credentials

2017/02/28

|| 02/05/2023 12

LectureSight is an open source real-time video tracking solution for PTZ cameras

Java, OpenCL, GPLv2 license, >22K lines of code. Created by Benjamin Wulff.

http://lecturesight.org (wiki, code, issue tracker, mailing list)

To run LectureSight, you need:• a Linux Capture Agent with a GPU (NVIDIA, Intel)• an overview camera (for example a webcam)• a PTZ camera which supports VISCA (Sony), VAPIX (Axis) or ONVIF

LectureSight

|| 02/05/2023 13

|| 02/05/2023 14

LectureSight:Overview image> detect changes > detect objects> direct camera (track objects)> steer the camera> send commands to PTZ camera (IP / serial)

Capture Agent:Record the PTZ camera video (IP / SDI / HDMI)

How does LectureSight work?

|| 02/05/2023 15

Why? Blackboards, body language, gestures

Running in production in 21 venues

Hardware: PTZ cameras: Axis V5915 (VAPIX), older Vaddio ClearView HD-USB (VISCA) Overview cameras: Logitech C930 webcam (USB), Raspberry Pi Cameras (IP) USB active extension cables (up to 20m), ATEN UCE-260 USB 2.0 over CAT5 (up to 60m) Ubuntu 16.04 Capture Agents running Galicaster 2.x with Nvidia GPUs (NV310+)

LectureSight scheduling is integrated with the Opencast / Galicaster calendar: tracking starts and stops for scheduled recordings.

LectureSight at UCT: Installations

|| 02/05/2023 16

Loss of calibration (overview or PTZ camera is moved out of position: LS-173) Poor overview camera performance in low lighting conditions False positives in object detection Lack of object persistence (presenter is tracked intermittently) Inability to distinguish presenter from background movement

(projection on screen) Inappropriate camera operator strategies (“always follow” not always right)

Lecturesight: What can go wrong?

|| 02/05/2023 17

Lecturesight server: control multiple venues with one instance (LS-179)

Improve object tracking – student project (Fraunhofer / University of Bonn)

More capable camera operator (pan/tilt/zoom, wider range of strategies)

Improved GUI and easier calibration (LS-173)

Production metrics (LS-189)

LectureSight: What’s next

|| 02/05/2023 18

4th-year Computer Science Student Project in 2016(Charles Fitzhenry, Mohamed Tanweer Khatieb, Maximilian Hahn) http://www.track4k.co.za

Now an open source project (C++, OpenCV, ECL2)https://github.com/LectureTracking/trackhd(just over 4000 lines of code)

An Opencast workflow operation coming soon.

Three operations: Detect blackboards Track motion of presenter Virtual panning

Track4K: Natural presenter tracking in 4K video

|| 02/05/2023 19

Source: 50 minutes 3920x2160 video recorded to mp4 file from an Axis 1428 IP camera over rtsp with pyca, bitrate 5.6 Mbps, 30fps (2G size), no audio track

Track4K compiled and run on Ubuntu 16.04 i7 8-core system with 4G RAM

Runtime for all three operations to produce 1080p output file (728M):158 minutes (3.16 x real-time)

Memory usage approx 1.2G resident (2.5G virtual)

Track4K: Performance

|| 02/05/2023 20

Investigation into more advanced tracking and cinematography (Masters’ degree work by Charles and Tanweer)

Tilt and Zoom (or even techniques like cross-fade) Optimize performance Integrate as an Opencast workflow (UCT) Track4K tracking data as a hints file for Theodul 4K playback with zoom (?)

Fork or clone on github, run it on your videos, provide feedback Github project, submit pull requests! (ECL2-compatible code).

Track4K: What’s next

|| 02/05/2023 21

LectureSight or Track4K?

Solution Advantages Disadvantages

High quality full HD image of the area of interest

No additional processing required

Requires additional hardware

Initial calibration is tricky

Real-time tracking can go wrong

Very smooth, natural camera movement (almost no errors)

Low-cost(no additional hardware)

Post-processing required

Image quality may be lowerthan PTZ equivalent