嵌入式计算机系统 - sjtuyshen/courses/embedded/l8.pdf · 嵌入式计算机系统 lecture #8...

嵌入式计算机系统

Lecture #8

MeeGo Multimedia

内容来自于meego.com以及MeeGo相关公开教程

Outline

•  MeeGo Multimedia Module Overview

•  Qt Mobility Multimedia

•  GStreamer Framework

•  APIs in Qt Mobility Multimedia

Multimedia in MeeGo Architecture

Media Stack Architecture

5

Media Application

Qt-Mobility MeeGo Touch

framework

GStreamer Qt

Video Decoder

Video Encoder Graphics

SST audio engine

HW/Driver

Other core modules

Core OS

libva

Multimedia in MeeGo Platform API

•  GStreamer

•  PulseAudio

Multimedia in MeeGo API

Qt Mobility Overview •  Qt Mobility Project provides desirable functions for

mobile platform

•  Qt Mobility covers a collection of APIs or frameworks

Bearer Management Contacts Location Message Multimedia

Publish and subscribe

Qt Service Framework

Sensors API

System Info …

Qt Mobility Multimedia Module •  QtMobility’s media library will

•  Easily create applications with system media capabilities •  Cover the items of most direct interest to developers

•  playback •  recording •  playlist •  management •  metadata •  radio •  media editing •  TV •  transcoding •  streaming

Qt Mobility Multimedia Module

•  Advantage: •  Implement fundamental Multimedia functions with

minimal code •  A great deal of flexibility with the media source •  Many different codecs are supported 'out of the box'

Qt Mobility Multimedia Module

•  High Level APIS •  Encapsulate the ability to manipulate media stream at varying level

without detailed knowledge of media subsystem

QT Mobility Multimedia Service

QT Mobility Multimedia High Level APIs

Playback List

Audio Video

Recording Playback Radio .. Camera

QtMobility Multimedia Module

•  Service - access to low-level of Multimedia function •  QAudio*

•  QAudioDeviceInfo, QAudioInput, QAudioOutput •  Low level cross platform audio API •  Low latency notifications, simple game, audio data processing/visualisation.

•  QVideo* •  QAbstractVideoBuffer, QVideoFrame, QabstractVideoSurface •  Low level tools for passing video frames. •  Abstraction for video buffers and surfaces

QtMobility Multimedia Module

•  QT MMF supports rich backend: •  GStreamer •  Quicktime •  Directshow •  …

QT MMF

GStreamer Service Plugin

QGstreamerService

QuickTimeService Plugin

QuickTimeService MyService

What is GStreamer

•  A framework for creating media applications •  Pipeline based •  Not a player •  One of the most obvious uses is to build a media player

•  Main advantage •  Pluggable elements could be mixed to form arbitrary

pipelines to support various kinds of video/audio codecs •  Unified API for user to create pipelines and media players

GStreamer Framework

Concepts in GStreamer

•  Element

•  Pad

•  Bin

•  Pipeline

•  Communication Mechanism

Concepts in GStreamer-element •  Element

•  Element is the most important object in GStreamer, it is derived from GstObject

•  Create a chain of elements linked together and let data flow through

this chain of elements

Concepts in GStreamer-element

•  The way to create an element

GstElement *element;

element = gst_element_factory_make(“fakesrc”, “source”);

if(!element) {

g_print(“Failed to create element\n”);

return -1;

}


•  Using an element as a GObject GstElement *element; gchar *name; element = gst_element_factory_make(“fakesrc”, “source”); if(!element) { g_print(“Failed to create element\n”); return -1; }

//get the name of element g_object_get(G_OBJECT(element), “name”, &name, NULL); g_print(“The name of element is %s.\n”,name); g_free(name);


•  Linking elements GstElement *pipeline; GstElement *source, *filter, *sink; pipeline = gst_pipeline_new(“my-pipeline”); source = gst_element_factory_make(“fakesrc”, “source”); filter = gst_element_factory_make(“identity”, “filter”); sink = gst_element_factory_make(“fakesink”, “sink”); gst_bin_add_many(GST_BIN(pipeline), source, filter, sink, NULL); if(!gst_element_link_many(source, filter, sink, NULL)) { g_warning(“Failed to link element!”); }


•  Element States •  GST_STATE_NULL •  GST_STATE_READY •  GST_STATE_PAUSED •  GST_STATE_PLAYING

•  Setting the element state •  gst_element_set_state() •  gst_element_sync_state_with_parent()

Concepts in GStreamer-pad

•  Pad •  Pad is used to negotiate links and data flow between

elements in GStreamer •  Pad could be sink pad or source pad •  Pad can restrict the type of data that flows through it

Concepts in GStreamer-pad •  Pad GObject +----GstObject +----GstElement +----ffdemux_ogg Pad Templates: SRC template: 'video_%02d' Availability: Sometimes Capabilities: ANY SRC template: 'audio_%02d' Availability: Sometimes Capabilities: ANY SINK template: 'sink' Availability: Always Capabilities: application/ogg

GObject +----GstObject +----GstElement +----GstMpeg2dec Pad Templates: SRC template: 'src' Availability: Always Capabilities: video/x-raw-yuv format: { YV12, I420, Y42B } width: [ 16, 4096 ] height: [ 16, 4096 ] framerate: [ 0/1, 2147483647/1 ] SINK template: 'sink' Availability: Always Capabilities: video/mpeg mpegversion: [ 1, 2 ] systemstream: false

Concepts in GStreamer-pad •  An example of using Dynamic pads static void cb_new_pad(GstElement *element, GstPad *pad, gpointer data) {

gchar *name; name = gst_pad_get_name(pad); //setup a new pad link for the newly create pad …

} GstElement *demux; demux = gst_element_factory_make(“oggdemux”, “demuxer”); //listen for newly create pads g_signal_connect(demux, “pad-added”, G_CALLBACK(cb_new_pad), NULL);

Concepts in GStreamer-Bin&Pipeline

•  Bin & Pipeline •  A Bin is a container for a collection of elements. E.g. playbin •  A pipeline is a special subtype of a bin that allows execution of all of

its contained child elements. •  A pipeline is a top-level bin. As you set it to PAUSED or PLAYING

state, data flow will start and media processing will take place. •  GstPipeline is derived from GstBin


•  Pipeline example - a simple ogg player


•  Creating a bin and add elements GstElement *bin, pipeline, *source, sink; //create pipeline pipeline = gst_pipeline_new(“my_pipeline”); //create bin bin = gst_bin_new(“my_bin”); source = gst_element_factory_make(“fakesrc”, “ source”); sink = gst_element_factory_make(“fakesink”, “sink”); //add elements to the bin gst_bin_add_many(GST_BIN(bin), source, sink, NULL); //add the bin to the pipeline gst_bin_add(GST_BIN(pipeline), bin);

Concepts in GStreamer-Communications

•  Mechanism for communication and data exchange between the application and the pipeline •  buffers: sent between elements in the pipeline •  events: sent between elements or from the application to

element •  messages: posted by element and collected by

application •  queries: application or element queries information from

other element


•  Different communications flows in GStreamer pipeline


•  Example of using bus static GMainLoop *loop; static gboolean my_bus_callback(Gbus *bus, GstMessage *message,

gpointer data) { switch(GST_MESSAGE_TYPE(message)) { case GST_MESSAGE_ERROR: { … break; } case GST_MESSAGE_EOS: { … break; } default: break; }

}


•  Example of using bus

GstElement *pipeline; GstBus *bus; pipeline = gst_pipeline_new(“my_pipeline”); bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); gst_bus_add_watch(bus, my_bus_callback, NULL); loop = g_main_loop_new(NULL, FALSE); g_main_loop_run(loop);

Tools in GStreamer •  gst-inspect

•  Print info about a GStreamer plugin or element •  Example: gst-inspect oggdemux

•  gst-launch •  Build and run a GStreamer pipeline •  Example: gst-launch filesrc location=/home/child.ogg ! mad !

alsasink

Example of video player

Player->play() QServicePlugin

QGstremerService

Playbin2

Typefind/autoplug

Demuxer

Decoder

Filter

…

QT MMF API QT MMF Backend GStreamer Base 3rd party plugins

Developing Developed

Multimedia Module API Hierarchy

QObject

QMediaObject

QAudioCaptureSource

QCamera

QMediaImageViewer

QMediaPlayer

QMediaService QMediaControl

QCameraControl

QMediaPlayerControl

QMediaRecorderControl

QMediaPlaylist QMediaRecorder QVideoWidget

Multimedia Module API

•  Multimedia module consists of APIs related to •  audio •  video •  image •  camera

Audio

•  Classes related to Audio •  QAudioCaptureSource •  QAudioDeviceInfo •  QAudioEncoderControl •  QAudioEncoderSettings •  QAudioEndpointSelector •  QAudioFormat •  QAudioInput •  QAudioOutput

QAudioCaptureSource

•  Provides an interface to query and select an audio input endpoint

•  Get and Set audio input to use

QAudioCaptureSource •  Functions in QAudioCaptureSource

•  Qlist<QString> QAudioCaptureSource::audioInputs() const Returns a list of available audio inputs •  void QAudioCaptureSource::setAudioInput(const Qstring &name) Set the active input to name

QAudioCaptureSource

•  Example to use QAudioCaptureSource QAudioCaptureSource *audiosource = new QAudioCaptureSource; //get audio input device names Foreach(const QString &device,audiosource->audioInputs()) {

ui->deviceBox->addItem(device,QVariant(device)) } //set audio input device name audiosource.setAudioInput(boxValue(ui->codecBox).toString()); //set the audiosource to the recorder QMediaRecorder *capture = new QMediaRecorder(audiosource); capture->record();

QAudioEncoderSettings •  Provides a set of audio encoder settings int bitRate() Return the bit rate of the compressed audio stream in bits per second void setBitRate(int rate) Set the audio bit rate in bits per second

QString codec() Return the audio codec void setCodec(const QString &codec) Set the audio codec

QtMultimediaKit::EncodingQuality quality() Returns the audio encoding quality void setQuality(QtMultimediaKit::EncodingQuality quality) Set the audio encoding quality

QAudioEncoderSettings

•  Example to use QAudioEncoderSettings

QAudioEncoderSettings audioSettings;

audioSettings.setCodec(“audio/mpeg”);

audioSettings.setQuality(QtMultimediaKit::HighQuality);

audioSettings.setChannelCount(2);

aduioSettings.setEncodingMode(QtMultimediaKit::ConstantQualityEncoding);

//set the audio encoder setting when record

QMediaRecorder *capture = new QMediaRecorder(audiosource);

capture.setEncodingSettings(audioSettings);

QMediaRecorder

•  Be used for the recording of media content

•  Intended to be used with other media object such as QAudioCaptureSource, QRadioTuner

•  State of QMediaRecorder •  QMediaRecorder::StoppedState •  QMediaRecorder:: RecordingState •  QMediaRecorder:: PausedState

QMediaRecorder

•  Signals in class QMediaRecorder •  void durationChanged(qint64 duration) emitted when the duration of the recorded media has

changed •  void stateChanged(QMediaRecorder::State state) emitted when a media recorder’s state has changed •  void mutedChanged(bool muted) emitted when the muted state has changed •  void error(QMediaRecorder::Error error) emitted when an error has occurred

QMediaRecorder •  Functions in class QMediaRecorder

•  void record() start recording •  void stop() stop recording •  void pause() pause recording •  bool setOutputLocation(const Qurl &location) set the destination location of media content •  void setEncodingSettings(const QAudioEncoderSettings &audio) set the audio and video encoder settings

QMediaRecorder •  Example of using QMediaRecorder

Examples

•  Audioinput

•  audiooutput

11-8-18 Embedded Computer Systems 47


•  Multimedia module consists of APIs related to •  audio •  video •  image •  camera

Video

•  Classes related to video •  QMediaPlayer •  QMediaPlayerList •  QVideoWidget

QMediaPlayer

•  Properties in QMediaPlayer •  duration: the duration of the current media •  muted: the muted state of the current media •  playbackRate: the playback rate of the current media •  playlist: the media playlist being used by the player

object •  position: the playback position of the current media •  volume: the current playback volume

QMediaPlayer

•  Properties in QMediaPlayer •  media: the active media source being used by the

player object •  state: the media player’s playback state QMediaPlayer::StoppedState QMediaPlayer::PlayingState QMediaPlayer::PausedState

QMediaPlayer

•  Functions in QMediaPlayer •  void play() start or resuming playing the current source •  void pause() pause playing the current source •  void stop() stop playing and reset the play position to the

beginning

QMediaPlayer

•  Example to use QMediaPlayer

QMediaPlayer *player = new QMediaPlayer;

player->setMedia(QUrl::fromLocalFile(“/home/song.mp3”));

player->setVolume(50);

player->play();

QMediaPlaylist

•  Provide a list of media content to play

•  Used with other media object like QMediaPlayer or QMediaImageViewer

QMediaPlaylist

•  Functions in QMediaPlaylist •  bool addMedia(const QMediaContent &content) append the media content to the playlist •  bool insertMedia(int pos, const QMediaContent &content) insert the media content to the playlist at position pos •  int mediaCount() const return the number of items in the playlist •  void next() advanced to the next media content in playlist •  void previous() return to the previous media content in playlist

QVideoWidget

•  Provides a widget which presents video produced by a media object

QVideoWidget

•  Properties in QVideoWidget •  brightness: int the brightness of displayed video ranging between -100 and 100 •  contrast: int the contrast of displayed video ranging between -100 and 100 •  hue: int the hue of displayed video ranging between -100 and 100 •  saturation: int the saturation of displayed video ranging between -100 and 100 •  fullScreen:bool whether video display is confined to a window or is fullScreen

Example

•  Example to use QMediaPlayer with QMediaPlaylist and QVideoWidget

playlist = new QMediaPlaylist; playlist->addMedia(Qurl(“http://example.com/movie1.mp4”)); playlist->addMedia(Qurl(“http://example.com/movie2.mp4”));

player = new QMediaPlayer; player->setPlayList(playlist);

videoWidget = new QVideoWidget; player->setVideoOutput(videoWidget); videoWidget->show();

player->play();

Example

•  Videowidget

11-8-18 Embedded Computer Systems 59


•  Multimedia module consists of APIs related to •  audio •  video •  Image •  camera

Image

•  Classes related to Image: •  QMediaImageViewer

QMediaImageViewer

•  Properties in QMediaImageViewer •  elapsedTime: int time elapsed since the current image was loaded •  timeout: int time that an image is displayed for before moving to

the next image •  state: QMediaImageViewer::State playlist control state of a slide show

QMediaImageViewer

•  Example of using QMediaImageViewer with QVideoWidget

viewer = new QMediaImageViewer;

videoWidget = new QVideoWidget;

viewer->bind(videoWidget);

videoWidget->show();

QMediaImageViewer

•  Example of using QMediaImageViewer with QMediaPlaylist

playlist = new QMediaPlaylist; playlist->setPlaybackMode(QMediaPlaylist::Loop); playlist->addMedia(image1); playlist->addMedia(image2); playlist->addMedia(image3); view->setPlaylist(playlist); view->setTimeout(5000); view->show();


•  Multimedia module consists of APIs related to •  audio •  video •  Image •  camera

Camera

•  Provides a framework to use Camera device features when supported by the platform •  Take still photo •  Take video •  Basic settings including flash mode, zoom level etc

•  Classes related to Camera •  QCamera •  QCameraViewfinder •  QCameraImageCapture

Camera

•  using camera to capture an image

QCamera *camera = new QCamera; QCameraViewfinder *viewFinder = new QCameraViewfinder(); viewFinder->show(); camera->setViewfinder(viewFinder); QCameraImageCapture *imageCapture = new

QCameraImageCapture(camera); camera->setCaptureMode(QCamera::CaptureStillImage); camera->searchAndLock; imageCapture->capture(); camera->unlock();

Camera

•  using camera to record video

QCamera *camera = new QCamera;

QMediaRecorder *mediaRecorder = new QMediaRecorder(camera);

camera->setCaptureMode(QCamera::CaptureVideo);

camera->start();

mediaRecorder->record();

嵌入式计算机系统 - sjtuyshen/courses/embedded/l8.pdf · 嵌入式计算机系统 lecture #8...

Documents