www.novocaptis.com

WPSS Phase 2

Audio/Video Hardware & Software Architecture

Christos Papachristou, Dipl-Eng, MSc, PhD

www.novocaptis.com

Helmet attachment

• Components– Camera– Omnidirectional microphone– Microcontroller– Push-button– LED driver– Light LEDs– Headphone jack– Microphone jack

www.novocaptis.com

Helmet attachment

www.novocaptis.com

Helmet attachment

• Connection cable

USB D+ 90 Ω twisted pair

USB D-

5V 2 Amp

GND 2 Amp

Microphone shielded

Headphones shielded

www.novocaptis.com

Software

• Main processing module– Gumstix Overo FireSTORM COM

• TI DM3730 800Mhz Cortex-A8 ARM• TI C64xx DSP @ 660Mhz• TPS65950 Audio and power management

chip• Marvell 88W8686 WiFi/Bluetooth module• 512Mb low power RAM• 512Mb NAND flash

www.novocaptis.com

Software

• Ångstrøm Linux– All packages are cross compiled using

the OpenEmbedded framework• Software modules are described through

recipes: Text files containing instructions on where to get the source, how to patch it, compile it and install it. It also defines its dependences on other software.Upon issuing of a build command, all dependencies are sorted out and compiled if necessary.

www.novocaptis.com

Software

• Ångstrøm Linux– All packages are cross compiled using

the OpenEmbedded framework• A recipe can also be a definition of a whole

image of an embedded system. All patches and changes in the software packages performed during phase 1 are described in a set of such recipes. Compiling of the complete operating system takes initially 4-5 hours and is accomplished by a single command. Incremental builds take much less.

www.novocaptis.com

Software

• Ångstrøm Linux– All packages are cross compiled using

the OpenEmbedded framework• The environment consists of two branches:

One maintained by the OpenEmbedded or related communities and one which has user additions. Software developed during phase 1 has been maintained as such a branch in an SVN repository. In phase 2 this is scheduled to be maintained in git.

www.novocaptis.com

Software

• Main software patches during phase 1• Kernel 2.6.34

– Power management functions of the TPS65950 chip

– OTG USB power override– Camera sensor driver

• Linphone– Audio buffer optimisation for TPS65950

• Gstreamer plugins– Camera drivers

www.novocaptis.com

Software

• Main services (ARM)• OpenVPN

– For static IP acquisition to override CERN wireless device policies.

• Wireless network daemon– Set of scripts to handle reconnections during

handover or temporal loss of signal

www.novocaptis.com

Software

• Audio services (ARM)• Linphone daemon• Command script

– A bash script that watches the state of the GPIO pins connected to the buttons assigned to the following actions:

Volume Up Sends command to ALSA mixer

Volume Down Sends command to ALSA mixer

Call/Hang UpSends command to Linphone

daemon

www.novocaptis.com

Software

• Audio services (Server)• Asterisk server

– Telephone book register between the mobile clients. Possible connection with CERN telephone network.

• Audio services (Supervision post)• Linphone or other VoIP solution

www.novocaptis.com

Software

• Streaming and Receiver services– Gstreamer pipeline

• The gstreamer libraries provide APIs both for prototyping and production code. During phase 1 only prototyping code was used in order to test the various pipeline elements and select among the most suitable for the specifications of the WPSS.

www.novocaptis.com

Software

• Streaming and Receiver services– Gstreamer pipeline

• Flexibility is quite limited in the prototyping APIs and particularly regarding to reconnections, which happen often in the ATLAS cavern.

• The services are currently coded in C++ and maintained in a git repository

www.novocaptis.com

Software

• Video services– Streaming service (ARM)– Receiver service (Server)– Transcoding service (Server)

www.novocaptis.com

Software

www.novocaptis.com

Software

www.novocaptis.com

Software

• Streaming service (ARM)– Gstreamer pipeline

– Video4Linux2 source element» Handling of Video4Linux2 IOCTLS for

exposure– Colour conversion element

» Employment of the ISP or DSP to handle more than 10-15 fps.

– H264 encoding element» DSP code to achieve better than 752x532

resolution. – Packetisation and transmission element

» Robust acknowledged communication

www.novocaptis.com

Software

• Receiver service (Server)– Gstreamer pipeline

– Reception and depacketisation element– H264 decoding element – Image processing/overlaying element– Input selector element– Video4Linux sink element

www.novocaptis.com

Software

• Transcoding service (Server)– VideoLan Client server

• Input from video loopback device• Transcoding into

– MJPEG– H264 in MPEG4 TS

» served as a file through HTTP» as an HTML video stream, after being split

into 10 second chunks

www.novocaptis.com

Software

• Transcoding service (Server)– VideoLan Client server

• The MJPEG stream is low delay and high bandwidth which better suits the needs of the supervisor post

• H264 has higher delay and low bandwidth and is better suited for storage.

– A video file is constantly output by the server. An external application can get it through HTTP and store it if needed for archiving purposes

– The HTML video streaming service provides a comparable capability

www.novocaptis.com

Software

• Implementation roadmap• Baseline system with main components in

C++– Production level code requires much more error-

correction capabilities, in order to be maintenance free and robust

– Faster performance

• Colour conversion, H264 encoding/decoding can be optimised on the hardware. Because of the modular design, they can be dealt with after the baseline system has been validated

www.novocaptis.com

Software

Thank you