Media Player for the i.MX31

Advanced Embedded Systems Architecture Class Project

May 14, 2011

Rafael CastroRyan UglandCarlos Cabral

Problem Definition/Goal

The goal of our project was to build a media player application using the i.MX31 board that could play mp3 files.

May 14, 2011 2Univ. of Texas at Austin

Current State of the Art Many different media

players exist today in different forms (phones, tablets, dedicated mp3 players, cars stereos, etc.)

We did not expand on the current state of the art, rather we were exploring the process and challenges involved in creating a media player app and running it on existing hardware.

May 14, 2011 Univ. of Texas at Austin 3

Freescale i.MX31 PDK

• 3-Stack Platform (CPU, Personality and Debug boards)• i.MX31 ARM 11 Processor• 128 MB DDR SDRAM• 256 MB NAND Flash• Audio• 2.7” TFT Display• Touch Controller

May 14, 2011 4Univ. of Texas at Austin

Project Definition and Tasks Setup Build Environment (Everyone)

◦ Installing and building LTIB for generating base Linux image for board◦ Setting up rootfs through NFS/TFTP for accessing file system on host

machine

Setup multi-media packages necessary for the player application (Everyone, but Rafael succeeded so we used his system)◦ Gstreamer and codecs◦ DirectFB graphics library◦ GUI Packages (Glade, Qtopia, GTK+, etc.)

Develop GUI Application (Ryan)◦ DirectFB GUI with buttons

May 14, 2011 Univ. of Texas at Austin 5

System Block Diagram

May 14, 2011 Univ. of Texas at Austin 6

i.MX31 3-Stack Board

Host Linux Machine

Serial-Console Messages

Ethernet-File System (rootfs)-Communication (NFS/TFTP)

Setup Build Environment

Linux Target Image Builder (LTIB)◦Generate Linux image for embedded system

NFS (Network File System) and Rootfs◦Embedded system accesses the rootfs on the Linux

host machine via NFS

TFTP (Trivial File Transfer Protocol)◦Used to load kernel image

May 14, 2011 Univ. of Texas at Austin 7

LTIB (Linux Target Image Builder)Open source project

that provides a utility to develop and deploy BSP’s (Board Support Package) for various target platforms

Final result is an image file that is transferred to to the embedded system (i.MX31)

May 14, 2011 Univ. of Texas at Austin 8

LTIB Image Configuration

May 14, 2011 Univ. of Texas at Austin 9

Many Issues During LTIB Compile!

The LTIB image generation is NOT a simple “click and go” process.

Various compile problems:◦Missing packages (.deb vs. .rpm packages)◦Broken scripts◦Incompatible versions◦Cache of intermediate state (difficult to debug)◦Permissions problems

May 14, 2011 Univ. of Texas at Austin 10

NFS (Network File System)Allows a user on a client computer to access files

over a networkWas used to allow the i.MX31 embedded system to

access the root file system (rootfs) on the host Linux machine

Preserves flash since limits writes to the flash. Changes to the rootfs are done on the Linux host

May 14, 2011 Univ. of Texas at Austin 11

TFTP (Trivial File Transfer Protocol) Simple file transfer protocol generally used for automated

transfer of configuration or boot files between machines in a local environment.

Was used to transfer the kernel image from the host Linux machine to the i.MX31 embedded system

May 14, 2011 Univ. of Texas at Austin 12

Multi-media PackagesAll open source packages for LinuxMany ports for embedded platforms availableEnabled different options for app.

development Difficult to setup. Easy to use!

◦GStreamer◦Tslib◦GTK+◦Glade3◦Qtopia

May 14, 2011 Univ. of Texas at Austin 13

GStreamerFramework for creating streaming media

applications (audio, video, etc)Provides:

◦ an API for multi-media applications◦a Plugin architecture◦a pipeline architecture (to connect plugins)◦a mechanism for media type handling/negotiation

Port of GStreamer to IMX31 available from Freescale

May 14, 2011 Univ. of Texas at Austin 14

GStreamer Example

May 14, 2011 Univ. of Texas at Austin 15

Example: gst-launch filesrc location=Kaleidoscope.avi! mfw_avidemuxer ! mfw_mpeg4decoder ! mfw_v4lsink

http://www.embeddeddesignindia.co.in/STATIC/PDF/201003/EDIOL_2010MAR16_MCP_AN_01.pdf?SOURCES=DOWNLOAD

TSLibTslib is an abstraction layer for touchscreen

panel events, as well as a filter stack for the manipulation of those events

Includes routines for finding and applying calibration values

Calibration values stored in file (pointed by TSLIB_CALIBFILE env. variable)

Source code found at: https://github.com/kergoth/tslib

May 14, 2011 Univ. of Texas at Austin 16

TSLibCalibration is very

important!!Couldn’t get GTK+

to read calibration values

Works with Qtopia

May 14, 2011 Univ. of Texas at Austin 17

GTK+Framework for creating graphical user

interfaces with cross platform compatibility.◦Glib: Main C library for graphical environment.

Provides data structures and API to create graphics and handle events.

◦Pango: A library for layout and rendering of text◦Cairo (2D graphics), ATK (accessibility) etc…

Cross-platform means you can develop in your PC and then deploy to embedded target

May 14, 2011 Univ. of Texas at Austin 18

Glade3Glade3 is a RAD (Rapid Application

Development) tool for designing GTK+ applications.

Basically:◦Draw user interface. ◦Define callbacks for specific events◦Creates XML file (“glade file”)◦Compile program◦Run, using Libglade library

May 14, 2011 Univ. of Texas at Austin 19

Glade3, GTK+

May 14, 2011 Univ. of Texas at Austin 20

Glade Interface ->

<- Example of GTK+ Text editor

Nice tutorial: http://www.micahcarrick.com/gtk-glade-tutorial-part-1.html

Qtopia (aka Qt …)Full environment for Mobile/PDKImplements virtual machine with:

◦Own virtual file system◦Own windowing system◦Java integration…

Built on top of GTK+, TSLib, GStreamer…Has changed names, groups etc.

◦Currently a version supported by Nokia (Qt)

May 14, 2011 Univ. of Texas at Austin 21

QtopiaPort of Qtopia

available for IMX31Includes a Media

Player!Made it work in our

system.Decided to develop

our own project outside Qtopia

Source: http://qt.nokia.com/

May 14, 2011 Univ. of Texas at Austin 22

DirectFB Overview

DirectFB is a an open source software library that provides access to the linux framebuffer.

Built in support for touchscreens and keyboards.

Used in Palm / HP WebOS.GTK+ apps can be run through DirectFB

rather than X-Windows on embedded devices.

Support for OpenGL

May 14, 2011 Univ. of Texas at Austin 23

DirectFB Media Player

Displays MP3 file list to screen

Keypad up and down keys used to select MP3

Keypad enter key used to play selected MP3

Images can also be displayed to screen

May 14, 2011 Univ. of Texas at Austin 24

DirectFB Media Player DevelopmentCoded in CUsed DirectFB for displaying MP3 names and

graphics to screenGUI developed on host machine using X-

Windows emulation of Linux framebuffer, and then recompiled on target machine.◦Only compile flags and media paths needed to be

changed between host and target machine compilesGstreamer library and codecs used for playing

the MP3 files

May 14, 2011 Univ. of Texas at Austin 25

Issues with DirectFB GuiDirectFB keyboard support did not work with

I.MX31 keypad.Touchcreen calibration did not work with

DirectFB.Decided not to use touchscreen controls.Disabled DirectFB keyboard support, and

accessed the keypad driver directly◦/dev/input/event0

Dumped all of the keypad event data to the terminal and determined how to uniquely identify each key.

May 14, 2011 Univ. of Texas at Austin 26

Results and Further WorkWe were able to successfully create a media

player application to play mp3’s.

The i.MX31 PDK has many features such as support for various media codecs and a touch screen LCD.◦Additional goals can be to allow the app to play

various media (ex. mpeg files)◦Further development of the GUI

Advanced menus and eye candy◦Utilize the touch screen

May 14, 2011 Univ. of Texas at Austin 27

References Freescale i.MX31 PDK Product Website for Documentation, Source Code and

Codecshttp://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=i.MX31PDK

◦ IMX_PDK15_LINUX_BSP (Board Support Package Download)◦ PDK 1.5 i.MX35 Linux User’s Guide◦ PDK 1.5 i.MX31 Linux Reference Manual◦ Linux Image Target Builder (LTIB) Documentation

“Building a Simple 3D Media Player for the i.MX31 in Linux”, Application Note by Multimedia Applications Division, Freescale Semiconductor, Inc., Austin, Texas.

Freescale Online Forums◦ http://forums.freescale.com/

Timesys LinuxLink Development Center for Freescale i.MX31◦ http://www.timesys.com/embedded-linux/resources/dev-center/imx31

Various internet sources for general Linux reference on topics such as package management, NFS, and TFTP server installation and setup.

May 14, 2011 Univ. of Texas at Austin 28