Media Tools for Matlab1 Help This document describes the objects and functions provided with the Media Tools for Matlab. These consist of: Matlab Objects:

mtplayer for playing audio/video files mtmovie for playing audio/video Matlab buffers mtreader for reading audio/video files mtwriter for writing audio/video files

Matlab Functions:

MediaToolsOutputDevices for determining the possible audio output devices listActiveMediaToolsInstances for listing all active Media Tools instances clearAllMediaToolsInstances for clearing and deleting all Media Tools instances

In order to use the Media Tools for Matlab, the Matlab path must first be properly configured. Please read the Media Tools for Matlab Setup Guide to see how this is done.

1 The Media Tools for Matlab have been developed with the support of the Army Hearing Program and the Walter

Reed National Military Medical Center.

mtplayer Create a Media Tools for Matlab player object for playing audio, video, and audio/video files.

Syntax When using default audio device:

mtp = mtplayer(FILENAME); When using audio device specified by Matlab output device ID:

mtp = mtplayer(FILENAME, ID); When using audio device specified by Media Tools for Matlab OUTPUT_DEV:

mtp = mtplayer(FILENAME, OUTPUT_DEV); When specifying initialization properties:

mtp = mtplayer(..., 'Property1', Property1Value , 'Property2', Property2Value, ...);

Description mtplayer creates a player for audio, video, and audio-video files. Files with audio and video content may be opened as audio-only or video-only, if desired. In the case of audio-only files or audio-video files opened as audio-only, no display is needed or created. This player can accommodate a variety of AV containers and codecs, including (for audio) AAC, MP3, WMA and uncompressed PCM and (for video) H.264, WMV, and uncompressed RGB. Instances of mtplayer may be created in the following ways: mtp = mtplayer(FILENAME) creates an instance of mtplayer for the file FILENAME.

FILENAME may be empty -- see below. mtp = mtplayer(FILENAME, ID) creates an instance of mtplayer for the file FILENAME

with audio played from output device ID. mtp = mtplayer(FILENAME, OUTPUT_DEV) created an instance of mtplayer for the file

FILENAME with audio played from the output device OUTPUT_DEV (as determined using the MTPLAYEROUTPUTDEVICES function).

mtp = mtplayer(...,'Property1', Property1Value, 'Property2', Property2Value, ...)

may be used to create instances of mtplayer analogous to those described above but with various player properties (see below) initialized at startup including parent figure window handle, display position within the figure, etc.

Unless otherwise specified, all audio will play over the default audio device and the video display (if required) will fill the current figure window (if no figure exists, one will be created).

Input Arguments FILENAME = the name of a valid audio, video, or audio/video file (e.g., *.wav, *.m4a,

*.wma, *.mp3, *.avi, *.m4v, *.mp4, *.mov, *.wmv, etc.). If FILENAME is an audio-only file, then no video display is created. If FILENAME is empty, then a blank video display according to either the default or startup properties (if specified).

ID = audio output device ID as returned by the native Matlab utility audiodevinfo. Audio

played using devices specified by ID will use Matlab's native audio interface API (typically DirectSound). ID = -1 corresponds to the default audio device.

OUTPUT_DEV = audio output device as specified by the Media Tools for Matlab utility

MediaToolsOutputDevices. Audio played using devices specified by OUTPUT_DEV may be played using the MME, DirectSound, WASAPI, and ASIO APIs.

'Property' = a string containing a valid, settable mtplayer property (see below) PropertyValue = an appropriate value for the associated Property.

Initialization-Only Properties The following properties may only be specified at the initialization of an mtplayer instance. 'ParentFigureHandle' = the handle to the Matlab figure in which the display will be

created. DEFAULT: If If not specified at initialization, the video display, if necessary, will be created in the current figure (if it exists) or in a figure created by mtplayer. If no display is necessary, then no figure is used or created.

'UseFFMPEG' = ['false','true'] may be used to configure the toolset used by mtplayer to

interact with AV files. This property may be set to: o 'false' to use the native Windows Media Foundation tools included with Microsoft

Windows 7 or beyond, or o 'true' to use the free FFMPEG toolset. This toolset is not included with mtplayer.

Instead, the interested user must manually download the relevant files. These are available at no cost from a variety of online resources including

http://ffmpeg.zeranoe.com/builds/. The necessary files may be found in the 32- or 64-bit 'Shared' builds:

http://ffmpeg.zeranoe.com/builds/win32/shared/ http://ffmpeg.zeranoe.com/builds/win64/shared/

It is recommended that the user download the version dated 29-Apr-2014, since this is the latest version with which the Media Tool for Matlab have been tested. Once downloaded, the files may be extracted using 7-Zip (which is available at SourceForge: http://sourceforge.net/projects/sevenzip/files/7-Zip/9.22/). After extraction, copy the following eight files from the extracted 'bin' folder into the appropriate 32- or 64-bit 'Media Tools for Matlab' install folder (typically C:\ProgramFiles\Sensimetrics\Media Tools For Matlab\Win32 or C:\ProgramFiles\Sensimetrics\Media Tools For Matlab\x64):

avcodec-55.dll avdevice-55.dll avfilter-4.dll avformat-55.dll avutil-52.dll postproc-52.dll swresample-0.dll swscale-2.dll

Please note that all Media Tools classes rely on the same underlying library, then if one class is initialized with 'UseFFMPEG' = 'true', then all other classes must also be initialized with 'UseFFMPEG' = 'true. The same applies for 'UseFFMPEG' = 'false'.

DEFAULT: 'false'.

Read-Write Properties

The following properties are available to the user both for reading and writing. 'AutoResize' = ['on','off'] may be used to specify whether the display will automatically

resize (preserving the same normalized DisplayPosition) as the window is resized. If set to 'off', then the display size as well as the horizontal offset from the left side of the figure and the vertical offset from the bottom of the figure will remain the same. DEFAULT: 'on'

'AVFilename' = the name of the AV file associated with the player.

DEFAULT: Set to FILENAME at initialization

'CurrentTime' = the current playback time. DEFAULT: Set to 0 = beginning of file

'DisplayPosition' = [XOFFSET, YOFFSET, WIDTH, HEIGHT] may be used to specify the

display position (in PIXELS) within the current figure window relative to the lower left corner. If any portion of the specified position is either outside the figure or overlapping with another mtplayer display, then this will return an error. DEFAULT: create a display that fills the current figure window and that uses letter- or pillar-boxing as specified by the LetterPillarBox property below

'EnableAudio' = ['on','off'] may be used to turn the audio on or off. This allows a video

file to be played in 'video-only' mode. If the file associated with the mtplayer has no audio content (i.e., it is video only), then this is ignored. DEFAULT: 'on'

'EnableVideo' = ['on','off'] may be used to turn the video display on or off. This allows a

video file to be played in 'audio-only' mode. If the file associated with the mtplayer has no video content, then this is ignored. DEFAULT: 'on'

'LetterPillarBox' = ['on','off'] may be used to turn letter- or pillar-boxing on or off in the

video display. If 'on', then the proper video aspect ratio will be maintained and black bars will appear at the top/bottom or left/right as necessary to fill the rest of the display. If 'off', then the video will stretch to fill the entire display. DEFAULT: 'on'

'LoopAVSource' = ['on','off'] may be used to enable or disable looping file playback. If

'on', then file playback will loop and repeat until manually stopped. If 'off', then file playback will stop when the end of the file is reached. DEFAULT: 'off'

'PlaySpeed' = the playback speed. If set to 1.0, then the playback speed is normal. If >

1.0, then playback is faster than normal. If < 1.0, then playback is slower than normal. Basically, this means that the video frames will be displayed at a rate of PlaySpeed*VideoFPS and audio samples will be played at a rate of PlaySpeed*AudioFs. Please note that there no attempt is made to interpolare the adjusted video frames to match the hardware rate of the display in use (e.g., 59.94 Hz). The existing frames are simply sent to the hardware at a faster or slower rate. Further, no attempt is made to

compensate the audio for pitch shifts that arise due to adjustment of the playback sampling rate. DEFAULT: 1.0

Advanced Read-Write Properties

The following properties expose advanced features of mtplayer that might be of interest to specialized users of mtplayer and are available for reading and writing.

'AlternateAudio' = ALTAUDIO = a structure containing audio content that will play out

over the mtplayer audio output device instead of the file audio. This is useful if it is desired to play the video with audio other than that from the original file. For example, a user might load the file audio (using the 'readAV' method associated with mtreader), process the audio in Matlab (e.g., filtering), and then use the processed audio for AV playback. The structure ALTAUDIO contains one or possibly two fields:

o 'ALTAUDIO.data' = the audio content channels stored column wise and at the same

sampling rate as the file audio (as indicated by the mtplayer property 'AudioFs'). o 'ALTAUDIO.start_time' = (OPTIONAL) the position within the AV file at which the

audio is inserted. Must be between 0 and the file duration (indicated by the mtplayer read-only property 'Duration'). The end time of the inserted audio will be inferred as being equal to: start_time + number-of-samples/sampling-rate. If the start time is not specified, a start time of 0 is assumed.

NOTE when using AlternateAudio: AlternateAudio effectively replaces the audio in the original file, which leads to the following behavior:

o The time-span of the file is restricted to the time span of the alternate audio as

specified by the read-only mtplayer properties AlternateAudioStartTime and AlternateAudioEndtime (which are automatically populated when the AlternateAudio value is assigned). This means that if the alternate audio covers only a subset of the duration of the original file and if source looping is active, then the looping will occur only over the time span of the alternate audio.

o The number of audio channels for the file effectively becomes the number of audio

channels in the alternate audio. For example, if the file has stereo audio and the alternate audio has 6 channels, then the mtplayer will consider the source to have 6 channels for as long as the alternate audio is active. This number of channels is reflected in the read-only AlternateAudioNumberChannels property. The setting of the SourceToOutputAudioMixing property (see below) will reflect this alternate number of channels.

If AlternateAudio is set to the empty value [], then the inserted audio will be cleared and audio will revert to the original file audio.

DEFAULT: [] = no Alternate Audio used

'SourceToOutputAudioMixing' = [NUM_SOURCE_CHANNELS-by-

NUM_DEVICE_CHANNELS] matrix of source-channel-to-output-device-channel audio mixing, where the source is _either_ the file audio or the user-supplied alternate audio -- depending upon which one is currently active. By default, the source channels are mapped on a one-to-one basis to the audio device output channels OR, in the special case of mono source audio and stereo output audio, the single audio channel will map to both output channels. It is possible to change this routing to suit the custom needs of an application. For example, it is possible to play stereo source audio over device channels 5 and 6, if desired. Further, it is possible to route the same source channel to multiple output device channels or to route multiple source channels to the same output device channel. Specifically,

SourceToOutputAudioMixing(i,j) = scale

describes the routing of source channel i, multiplied by the factor 'scale', to output device channel j. For example,

SourceToOutputAudioMixing = [0 0 1 0;

0 0 0 1]

routes stereo source audio to output device channels 3 and 4. Similarly

SourceToOutputAudioMixing = [0.5 0 1 0; 0.1 0 0 2]

plays 0.5*(source channel 1) + 0.1*(source channel 2) to output channel 1 while simultaneously playing 1*(source channel 1) to output channel 3 and 2*(source channel 2) to output channel 4. DEFAULT: For monaural audio, send the single audio channel to both channels of a stereo output device and for multi-channel audio, send audio channel CH to output channel CH.

'VideoDelayReAudioMsec' = DELAY may be used to delay (>0) or advance (<0) the video

component of the playback relative to the audio. The value is specified in milliseconds. DEFAULT: 0

'VideoDisplayZones' = VIDEOZONES = a structure describing video playback zones. This

permits the user to override standard video playback to force the display to show either a solid black screen or a random noise screen instead of the normal file video for specific time periods during playback. VIDEOZONES has two fields:

o ' VIDEOZONES .times' = a vector of N+1 times specifying the start and stop times for N

difference playback zones. Specifically, VIDEOZONES .times([n, n+1]) specify the start and stop times of zone 'n' for n = 1, 2, ..., N. All times must lie within the content duration (i.e., between 0 and 'Duration').

o ' VIDEOZONES .types' = a vector of N values that describe the zone type: 0 = 'normal playback', 1 = 'black-screen playback', and 2 = 'random-noise-screen playback'.

If VIDEOZONES.times(1)>0, then an implicit 'normal' zone spanning from 0 to time VIDEOZONES times(1) is assumed. Similarly, if VIDEOZONES.times(N+1) < 'Duration', then an implicit 'normal' zone spanning from VIDEOZONES.times(N+1) to 'Duration' is assumed. If VideoDisplayZones is set to the empty value [], then all video zones are cleared and all video will play normally. DEFAULT: [] = normal playback

Read-Only Properties The following properties are available for read only. 'AlternateAudioNumberChannels' = number of channels of altenate audio (if in use). 'AlternateAudioStartTime' = Start time of alternate audio relative to the start of file. 'AlternateAudioEndTime' = End time of alternate audio relative to the start of file. 'AudioAPI' = API used for audio playback ('MME', 'DirectSound', 'WASAPI' or 'ASIO'. 'AudioDeviceID' = output audio device identifier ID (see audiodevinfo) if specified upon

mtplayer initialization and -1 for default audio device. 'AudioDeviceName' = name of audio device. 'AudioFs' = sampling rate of audio in Hz. 'AudioNumberChannels' = number of channels of source audio. 'Duration' = the duration (in seconds) of the file. 'IsPlaying' = flag that returns 1 if the file is currently playing and 0 if it is stopped or

paused.

'ParentFigureSize' = size (in pixels) of the parent figure in which the display resides. 'VideoFPS' = frame rate of the video in frames per second. 'VideoPixelWidth' = width of video source in pixels. This may be different than the width

of the video display. 'VideoPixelHeight' = height of video source in pixels. This may be different than the

height of the video display.

Methods get(mtp,'Property')

Query value of mtplayer read-write or read-only 'Property'.

isplaying(mtp)

Returns 1 if playback is active and 0 if playback is paused.

pause(mtp)

Pause playback.

play(mtp), play(mtp,start), play(mtp,[],stop), play(mtp,start,stop)

Play content. Optional start and stop times may be specified. If start is omitted, then playback will commence from the current play point (i.e., 'CurrentTime'). If stop is omitted, playback will continue until the end of file has been reached unless 'LoopAVSource' has been set to 'on', in which case playback will loop indefinitely.

playblocking(mtp), playblocking(mtp,start), playblocking(mtp,[],stop),

playblocking(mtp,start,stop) Similar to 'play' except that control will not return to the Matlab command line until playback has been completed. 'LoopAVSource' is ignored for blocking playback.

resume(mtp)

Restart playback from paused position.

set(mtp,'Property1',PropertyVal1,'Property2',PropertyVal2,...) Set values of valid, writeable mtplayer properties.

stop(mtp) Stop playback and return to beginning.

mtmovie Create a Media Tools for Matlab movie object for playing audio, video, and audio/video buffer content.

Syntax When using default audio device:

mtm = mtmovie(); mtm = mtmovie(A); mtm = mtmovie(V); mtm = mtmovie(A, V); mtm = mtmovie(V, A);

When using audio device specified by Matlab output device ID:

mtm = mtmovie(ID); mtm = mtmovie(A,ID); mtm = mtmovie(V,ID); mtm = mtmovie(A, V,ID); mtm = mtmovie(V, A,ID);

When using audio device specified by Media Tools for Matlab OUTPUT_DEV:

mtm = mtmovie(OUTPUT_DEV); mtm = mtmovie(A, OUTPUT_DEV); mtm = mtmovie(V, OUTPUT_DEV mtm = mtmovie(A, V, OUTPUT_DEV); mtm = mtmovie(V, A, OUTPUT_DEV);

When specifying initialization properties:

mtm = mtmovie(..., 'Property1', Property1Value, 'Property2', Property2Value, ...);

Description mtmovie creates a player for audio, video, and audio-video movies constructed using Matlab buffer content. In the case of audio-only movies no display is needed or created. Instances of mtmovie may be created in the following ways: mtm = mtmovie() creates an empty mtmovie. Content may be added to an mtmovie

using the appendAV method described below.

mtm = mtmovie(A) creates an mtmovie with audio content specified by the structure A. An mtmovie initialized with audio content only may have video added subsequently using the appendAV method described below.

mtm = mtmovie(V) creates mtmovie with video content specified by the structure V. An mtmovie initialized with video content only may have audio added subsequently using the appendAV method described below.

mtm = mtmovie(A, V) or mtm = mtmovie(V, A) creates an mtmovie with both audio and video content, where A and V are as described above.

mtm = mtmovie(ID), mtm = mtmovie(A, ID), mtm = mtmovie(V, ID),

mtm = mtmovie(A, V, ID), and mtm = mtmovie(V, A, ID) operate as described above with the extension that audio will be played from output device ID.

mtm = mtmovie(OUTPUT_DEV), mtm = mtmovie(A, OUTPUT_DEV), mtm = mtmovie(V, OUTPUT_DEV), mtm = mtmovie(A,V,OUTPUT_DEV), and mtm = mtmovie(V,A,OUTPUT_DEV) operate as described above with the extension that audio will be played from the output device OUTPUT_DEV (as determined using the MTPLAYEROUTPUTDEVICES function).

mtm = mtmovie (...,'Property1', Property1Value, 'Property2', Property2Value, ...)

may be used to create instances of mtmovie analogous to those described above but with various mtmovie properties (see below) initialized at startup including parent figure window handle, display position within the figure, etc.

Input Arguments A = structure specifying audio content. A must have two fields:

o 'A.data' contains the channels of audio content stored column-wise.

o 'A.Fs' contains the audio sampling rate.

If A contains additional fields, these are ignored by mtmovie.

V = structure specifying video content. V must have two fields:

o 'V.M' is an array of structures containing the video content. Specifically, [V.M(1),

V.M(2), ..., V.M(K)] specify K frames of video. Each V.M(k).cdata is a pix_height-by-pix_width-by-3 array of UINT8 values specifying the RGB values of the video frame. All K frames must be of equal dimensions. The R, G, and B values of each frame image occupy V.M(k).cdata(:,:,1), V.M(k).cdata(:,:,2), and V.M(k).cdata(:,:,3), respecively. The dimensions of the video content (in pixels) is inferred from the dimensions of V.M(k).

o 'V.fps' contains the video frame rate in frames per second. If V contains additional fields, these are ignored by mtmovie.

ID = audio output device ID as returned by the native Matlab utility audiodevinfo. Audio played using devices specified by ID will use Matlab's native audio interface API (typically DirectSound). ID = -1 corresponds to the default audio device.

OUTPUT_DEV = audio output device as specified by the Media Tools for Matlab utility

MediaToolsOutputDevices. Audio played using devices specified by OUTPUT_DEV may be played using the MME, DirectSound, WASAPI, and ASIO APIs.

'Property' = a string containing a valid, settable mtplayer property (see below) PropertyValue = an appropriate value for the associated Property.

Initialization-Only Properties The following property may only be specified at the initialization of an mtmovie instance. 'ParentFigureHandle' = the handle to the Matlab figure in which the display will be

created. DEFAULT: If not specified at initialization, the video display, if necessary, will be created in the current figure (if it exists) or in a figure created by mtmovie. If no display is necessary, then no figure is used or created.

'UseFFMPEG' = ['false','true'] may be used to configure the toolset used by mtplayer to interact with AV files. Even though the mtmovie class does not interact with AV file natively, other Media Tools classes (mtplayer and mtreader) do. Since all of the Media Tools classes rely on the same underlying library of code, if any active classes set 'UseFFMPEG' to 'true', then all classes must set 'UseFFMPEG' to 'true'. Please see the help for either mtplayer or mtreader for details on FFMPEG. DEFAULT = 'false'.

Read-Write Properties

The following properties are available to the user both for reading and writing. 'AudioFs' = the audio sampling rate in Hz. This property may NOT be specified during

mtmovie initalization, since it is specified as part of the initial audio content structure A. After initialization, however, the sampling rate may be changed.

DEFAULT: specified by the initial audio content structure A

'AutoResize' = ['on','off'] may be used to specify whether the display will automatically

resize (preserving the same normalized DisplayPosition) as the window is resized. If set

to 'off', then the display size as well as the horizontal offset from the left side of the figure and the vertical offset from the bottom of the figure will remain the same. DEFAULT: 'on'

'CurrentTime' = the current playback time.

DEFAULT: 0 = beginning of mtmovie

'DisplayPosition' = [XOFFSET, YOFFSET, WIDTH, HEIGHT] may be used to specify the

display position (in PIXELS) within the current figure window relative to the lower left corner. If any portion of the specified position is either outside the figure or overlapping with another mtmovie display, then this will return an error. DEFAULT: create a display that fills the current figure window and that uses letter- or pillar-boxing as specified by the LetterPillarBox property below

'EnableAudio' = ['on','off'] may be used to turn the audio on or off. This allows an

audio/video mtmovie to be played in 'video-only' mode. If the mtmovie has no audio content, then this is ignored. DEFAULT: 'on'

'EnableVideo' = ['on','off'] may be used to turn the video display on or off. This allows an

audio/video mtmovie to be played in 'audio-only' mode. If the mtmovie has no video content, then this is ignored. DEFAULT: 'on'

'LetterPillarBox' = ['on','off'] may be used to turn letter- or pillar-boxing on or off in the

video display. If 'on', then the proper video aspect ratio will be maintained and black bars will appear at the top/bottom or left/right as necessary to fill the rest of the display. If 'off', then the video will stretch to fill the entire display. DEFAULT: 'on'

'LoopAVSource' = ['on','off'] may be used to enable or disable looping mtmovie playback.

If 'on', then mtmovie playback will loop and repeat until manually stopped. If 'off', then mtmovie playback will stop when the end of the mtmovie is reached. DEFAULT: 'off'

'VideoFPS' = the video frame rate in frames per second. This property may NOT be

specified during mtmovie initialization, since it is specified as part of the initial video content structure V. After initialization, however, the frame rate may be changed.

DEFAULT: specified by the initial video content structure V

Advanced Read-Write Properties

The following properties expose advanced features of mtplayer that might be of interest to specialized users of mtmovie and are available for reading and writing.

'SourceToOutputAudioMixing' = [NUM_SOURCE_CHANNELS-by-

NUM_DEVICE_CHANNELS] matrix of source-content-channel-to-output-device-channel audio mixing. By default, the source channels are mapped on a one-to-one basis to the audio device output channels OR, in the special case of mono source audio and stereo output audio, the single audio channel will map to both output channels. It is possible to change this routing to suit the custom needs of an application. For example, it is possible to play stereo source audio over device channels 5 and 6, if desired. Further, it is possible to route the same source channel to multiple output device channels or to route multiple source channels to the same output device channel. Specifically,

SourceToOutputAudioMixing(i,j) = scale

describes the routing of source channel i, multiplied by the factor 'scale', to output device channel j. For example,

SourceToOutputAudioMixing = [0 0 1 0;

0 0 0 1]

routes stereo source audio to output device channels 3 and 4. Similarly

SourceToOutputAudioMixing = [0.5 0 1 0; 0.1 0 0 2]

plays 0.5*(source channel 1) + 0.1*(source channel 2) to output channel 1 while simultaneously playing 1*(source channel 1) to output channel 3 and 2*(source channel 2) to output channel 4. DEFAULT: For monaural audio, send the single audio channel to both channels of a stereo output device and for multi-channel audio, send audio channel CH to output channel CH.

'VideoDelayReAudioMsec' = DELAY may be used to delay (>0) or advance (<0) the video

component of the playback relative to the audio. The value is specified in milliseconds. DEFAULT: 0

'VideoDisplayZones' = VIDEOZONES = a structure describing video playback zones. This permits the user to override standard video playback to force the display to show either a solid black screen or a random noise screen instead of the normal file video for specific time periods during playback. VIDEOZONES has two fields: o ' VIDEOZONES .times' = a vector of N+1 times specifying the start and stop times for N

difference playback zones. Specifically, VIDEOZONES .times([n, n+1]) specify the start and stop times of zone 'n' for n = 1, 2, ..., N. All times must lie within the content duration (i.e., between 0 and 'Duration').

o ' VIDEOZONES .types' = a vector of N values that describe the zone type: 0 = 'normal playback', 1 = 'black-screen playback', and 2 = 'random-noise-screen playback'.

If VIDEOZONES.times(1)>0, then an implicit 'normal' zone spanning from 0 to time VIDEOZONES times(1) is assumed. Similarly, if VIDEOZONES.times(N+1) < 'Duration', then an implicit 'normal' zone spanning from VIDEOZONES.times(N+1) to 'Duration' is assumed. If VideoDisplayZones is set to the empty value [], then all video zones are cleared and all video will play normally. DEFAULT: [] = normal playback

Read-Only Properties The following properties are available for read only. 'AudioAPI' = API used for audio playback ('MME', 'DirectSound', 'WASAPI' or 'ASIO'. 'AudioDeviceID' = output audio device identifier ID (see audiodevinfo) if specified upon

mtmovie initialization and -1 for default audio device. 'AudioDeviceName' = name of audio device. 'AudioNumberChannels' = number of channels of source audio. 'AudioNumberSamples' = number of samples of source audio. 'Duration' = the duration (in seconds) of the mtmovie. 'IsPlaying' = flag that returns 1 if the mtmovie is currently playing and 0 if it is stopped

or paused. 'ParentFigureSize' = size (in pixels) of the parent figure in which the display resides.

'VideoNumberFrames' = number of video frames. 'VideoPixelWidth' = width of video source in pixels. This may be different than the width

of the video display. 'VideoPixelHeight' = height of video source in pixels. This may be different than the

height of the video display.

Methods appendAV(mtm, A), appendAV(mtm, V), appendAV(mtm, A, V), appendAV(mtm, V, A)

Append audio and/or video to the mtmovie. Additional audio and video are specified using the structures A and V, respectively, as described above. If audio is being appended to an mtmovie that already contains audio, then the number of channels and the audio sampling rate must match those of the existing audio. If video is being appended to an mtmovie that already contains video, then the video pixel dimensions and the video frame rate must match those of the existing video.

get(mtm,'Property')

Query value of mtmovie read-write or read-only 'Property'.

A = getA(mtm), A = getA(mtm, range)

Read audio content from the mtmovie. Audio is returned in a structure analogous to the A structure used to add audio to an mtmovie (with fields 'data' and 'Fs'). If the range is specified, it must be a two-element vector specifying the start and end times of the audio to read. The specified start time must be prior to the specified end time, and both times must be within 0 and the mtmovie 'Duration'.

V = getV(mtm), V = getV(mtm, range) Read video content from the mtmovie. Video is returned in a structure analogous to the V structure used to add video to an mtmovie (with fields 'M' and 'fps'). If the range is specified, it must be a two-element vector specifying the start and end times of the audio to read. The specified start time must be prior to the specified end time, and both times must be within 0 and the mtmovie 'Duration'.

[A,V] = getAV(mtm), [A,V] = getAV(mtm, range) Read audio and video content from the mtmovie. These combine the 'getA' and 'getV' methods.

isplaying(mtm) Returns 1 if playback is active and 0 if playback is paused.

pause(mtm)

Pause playback.

play(mtm), play(mtm, start), play(mtm, [], stop), play(mtm, start, stop)

Play content. Optional start and stop times may be specified. If start is omitted, then playback will commence from the current play point (i.e., 'CurrentTime'). If stop is omitted, playback will continue until the end of file has been reached unless 'LoopAVSource' has been set to 'on', in which case playback will loop indefinitely.

playblocking(mtm), playblocking(mtm, start), playblocking(mtm, [], stop),

playblocking(mtm, start, stop) Similar to 'play' except that control will not return to the Matlab command line until playback has been completed. 'LoopAVSource' is ignored for blocking playback.

resume(mtm)

Restart playback from paused position.

set(mtm,'Property1',PropertyVal1,'Property2',PropertyVal2,...) Set values of valid, writeable mtmovie properties.

setAV(mtm, A), setAV(mtm, V), setAV(mtm, A, V), setAV(mtm, V, A),

setAV(mtm, V, 'DisplayPosition', [XOFFSET, YOFFSET, WIDTH, HEIGHT]), setAV(mtm, A, V, 'DisplayPosition', [XOFFSET, YOFFSET, WIDTH, HEIGHT]), setAV(mtm, V, A, 'DisplayPosition', [XOFFSET, YOFFSET, WIDTH, HEIGHT]) Replace the audio and/or video content of an mtmovie. When replacing audio/video, the previous audio/video is entirely discarded and replaced with the new audio/video. When replacing only the audio, any existing video remains unchanged. Similarly, when replacing the only the video, any existing audio remaing unchanged. In the special cases where the video is being replaced, the DisplayPosition property may be optionally updated at the same time (to resize the display if desired). Audio and video are specified using the structures A and V, respectively, as described above. In the special case where A.data = [], all audio content is removed from the mtmovie. Similarly, if V.M = [], all video content is removed from the mtmovie.

stop(mtm)

Stop playback and return to beginning.

mtreader Create a Media Tools for Matlab reader object for reading audio, video, and audio/video files.

Syntax

mtr = mtreader(FILENAME); mtr = mtreader(FILENAME, 'Property1', Property1Value, 'Property2', Property2Value,

...);

Description mtreader creates a reader for audio, video, and audio-video files. This reader can accommodate a variety of AV containers and codecs, including (for audio) AAC, MP3, WMA and uncompressed PCM and (for video) H.264, WMV, and uncompressed RGB. Instances of mtreader may be created in the following ways: mtr = mtreader(FILENAME) creates an instance of mtreader for the file FILENAME. mtr = mtreader(FILENAME,'Property1', Property1Value, 'Property2', Property2Value, ...)

may be used to create instances of mtreader analogous to that described above but with various reader properties (see below) initialized at startup.

Input Arguments FILENAME = the name of a valid audio, video, or audio/video file (e.g., *.wav, *.m4a,

*.wma, *.mp3, *.avi, *.m4v, *.mp4, *.mov, *.wmv, etc.).

'Property' = a string containing a valid, settable mtreader property (see below) PropertyValue = an appropriate value for the associated Property.

Initialization-Only Properties The following property may only be specified at the initialization of an mtreader instance. 'UseFFMPEG' = ['false','true'] may be used to configure the toolset used by mtreader to

interact with AV files. This property may be set to: o 'false' to use the native Windows Media Foundation tools included with Microsoft

Windows 7 or beyond, or

o 'true' to use the free FFMPEG toolset. This toolset is not included with mtreader.

Instead, the interested user must manually download the relevant files. These are available at no cost from a variety of online resources including http://ffmpeg.zeranoe.com/builds/. The necessary files may be found in the 32- or 64-bit 'Shared' builds:

http://ffmpeg.zeranoe.com/builds/win32/shared/ http://ffmpeg.zeranoe.com/builds/win64/shared/

It is recommended that the user download the version dated 29-Apr-2014, since this is the latest version with which the Media Tool for Matlab have been tested. Once downloaded, the files may be extracted using 7-Zip (which is available at SourceForge: http://sourceforge.net/projects/sevenzip/files/7-Zip/9.22/). After extraction, copy the following eight files from the extracted 'bin' folder into the appropriate 32- or 64-bit 'Media Tools for Matlab' install folder (typically C:\ProgramFiles\Sensimetrics\Media Tools For Matlab\Win32 or C:\ProgramFiles\Sensimetrics\Media Tools For Matlab\x64):

avcodec-55.dll avdevice-55.dll avfilter-4.dll avformat-55.dll avutil-52.dll postproc-52.dll swresample-0.dll swscale-2.dll

Please note that all Media Tools classes rely on the same underlying library, then if one class is initialized with 'UseFFMPEG' = 'true', then all other classes must also be initialized with 'UseFFMPEG' = 'true. The same applies for 'UseFFMPEG' = 'false'.

DEFAULT: 'false'.

Read-Write Properties

The following properties are available to the user both for reading and writing. 'AVFilename' = the name of the AV file associated with the reader. Set to FILENAME at

initialization. 'CurrentTime' = the current read time.

'VideoScale' = scale factor (in percent) applied to the video size when reading video. Video output read from the file will be re-scaled to a size of:

VideoScale*[FileVideoPixelWidth FileVideoPixelHeight] pixels.

The main reason for this scaling is to permit better Matlab memory management. Specifically, if a source video is high resolution, then reading in multiple seconds of video can rapidly overwhelm the available Matlab memory. By scaling the video to be smaller, less memory is consumed.

VideoScale must be less than 100, with 100 being full-scale. The minimum value of VideoScale will depend upon the file and will be selected so that the minimum scaled dimension of a video frame is at least 2 pixels.

Read-Only Properties The following properties are available for read only. 'AudioFs' = sampling rate of audio in Hz. 'AudioNumberChannels' = number of channels of source audio. 'Duration' = the duration (in seconds) of the file. 'FileVideoPixelWidth' = width of video file source in pixels. 'FileVideoPixelHeight' = height of video file source in pixels. 'OutputVideoPixelWidth' = width of video output in pixels =

round(VideoScale*FileVideoPixelWidth). 'OutputVideoPixelHeight' = height of video output in pixels =

round(VideoScale*FileVideoPixelHeight). 'VideoFPS' = frame rate of the video in frames per second.

Methods get(mtr,'Property')

Query value of mtreader read-write or read-only 'Property'.

A = readA(mtr), A = readA(mtr, start), A = readA(mtr, [], stop), A = readA(mtr, start, stop)

Read audio content. Start and stop times may be specified. If start is omitted, reading will commence at CurrentTime. If stop is omitted, then audio will be read until the end-of-file. If the file has video content, it is ignored. Audio content is returned as a structure A with two fields: o 'A.data' contains the channels of audio content stored column-wise.

o 'A.Fs' contains the audio sampling rate.

V = readV(mtr), V = readV(mtr, start), V = readV(mtr, [], stop), V = readV(mtr, start, stop)

Read video content. Start and stop times may be specified. If start is omitted, reading will commence at CurrentTime. If stop is omitted, then video will be read until the end-of-file. If the file has audio content, it is ignored. Video content is returned as a structure V with three fields: o 'V.M' is an array of structures containing the video content. Specifically, [V.M(1),

V.M(2), ..., V.M(K)] specify K frames of video. Each V.M(k).cdata is a pix_height-by-pix_width-by-3 array of UINT8 values specifying the RGB values of the video frame. All K frames must be of equal dimensions. The R, G, and B values of each frame image occupy V.M(k).cdata(:,:,1), V.M(k).cdata(:,:,2), and V.M(k).cdata(:,:,3), respecively. The dimensions of the video content (in pixels) is inferred from the dimensions of V.M(k).

o 'V.times' is a vector that is the same length as V.M and that contains the timestamps for each frame of video.

o 'V.fps' contains the video frame rate in frames per second.

[A,V] = readAV(mtr), [A,V] = readAV(mtr, start), [A,V] = readAV(mtr, [], stop),

[A,V] = readAV(mtr, start, stop) Read audio and video content. Start and stop times may be specified. If start is omitted, reading will commence at CurrentTime. If the file is either audio-only or video-only, then the missing content (video or audio, respectively) is left unfilled.

set(mtr,'Property1',PropertyVal1,'Property2',PropertyVal2,...)

Set values of valid, writeable mtreader properties.

mtwriter Create a Media Tools for Matlab writer object for writing audio, video, and audio/video files.

Syntax

mtw = mtwriter(FILENAME); mtw = mtwriter(FILENAME, AUDIO_ENCODER); mtw = mtwriter(FILENAME, VIDEO_ENCODER); mtw = mtwriter(FILENAME, AUDIO_ENCODER, VIDEO_ENCODER); mtw = mtwriter(FILENAME, VIDEO_ENCODER, AUDIO_ENCODER); mtw = mtwriter (... , 'SuppressWarnings', WARNINGS_VALUE);

Description mtwriter creates a writer for creating audio, video, and audio-video files. This writer can accommodate a variety of AV containers and codecs, including (for audio) AAC and uncompressed PCM and (for video) H.264 and uncompressed RGB. Instances of mtwriter may be created in the following ways: mtw = mtwriter(FILENAME) creates an instance of mtwriter for the file FILENAME using

the default audio and video settings for the particular file container.

mtw = mtwriter(FILENAME, AUDIO_ENCODER) creates an instance of mtwriter for the file FILENAME using the specified audio settings and the default video settings.

mtw = mtwriter(FILENAME, VIDEO_ENCODER) creates an instance of mtwriter for the file FILENAME using the specified video settings and the default audio settings.

mtw = mtwriter(FILENAME, AUDIO_ENCODER, VIDEO_ENCODER) and mtw = mtwriter(FILENAME, VIDEO_ENCODER, AUDIO_ENCODER) create an instance of mtwriter for the file FILENAME using the specified audio and video settings.

mtw = mtwriter (... , 'SuppressWarnings', 'true'), mtw = mtwriter (... , 'UseFFMPEG', 'true'), mtw = mtwriter (... , 'SuppressWarnings', 'true', 'UseFFMPEG', 'true'), mtw = mtwriter (... , 'UseFFMPEG', 'true', 'SuppressWarnings', 'true') may be used to create instances of mtwriter analogous to those described above but with the 'SuppressWarnings' or 'UseFFMPEG' properties set to'true' to either disable overwrite-existing-file warnings or to configure the mtwriter for use with other Media Tools instances that use the FFMPEG library.

Input Arguments FILENAME = the name of the output file. The file extension will automatically determine

the type of file container:

*.wav = Audio-only WAV file *.m4a = Audio-only M4A (i.e., MP4 audio) file *.m4v, *.mp4 = Audio/Video or Video-only M4V/MP4 (i.e., MP4 video) file *.asf, *.wmv = Audio/Video or Video-only ASF/WMV file *.avi = Audio/Video AVI file (requires Windows 8.1)

File extensions that do not fall in this set will result in an error.

AUDIO_ENCODER = audio encoding information, described in a structure with two fields: o ' AUDIO _ENCODER.type' specifies the type of audio encoding to use. Currently,

mtwriter permits two forms of audio encoding, 'pcm' for uncompressed PCM encoding and 'aac' for compressed AAC encoding.

o EITHER ' AUDIO _ENCODER.bits' (for PCM encoding only) specifies the number of bits

per sample 16, 24, or 32.

OR ' AUDIO _ENCODER.kbps' (for AAC encoding only) specifies the target bit rate (related to the compression quality) 96, 128, 160, or 128.

VIDEO_ENCODER = video encoding information, described in a structure with either one

or two fields: o ' VIDEO _ENCODER.type' specifies the type of video encoding to use. Currently,

mtwriter permits two forms of video encoding, 'rgb' for uncompressed RGB encoding and h264 for compressed H.264 encoding.

o EITHER ' VIDEO _ENCODER.quality' (for H.264 encoding only) specifies the desired

video encoding quality as an integer ranging from 1 (lowest quality) to 100 (highest quality).

OR ' VIDEO _ENCODER.kbps' (also for H.264 encoding only) specifies the target bit rate (related to the compression quality) as a value spanning 256 (low quality) to 15000 (high quality) kbps. NOTE: for H.264 encoding, the compression quality may be controlled by either the 'quality' or 'kbps' value, but not both. It is up to the user to select the method that yields the desired results. NOTE: RGB encoding has no quality-related parameters.

'SuppressWarning' = an initialization property that, if set to 'true', will suppress any

existing-file-overwrite warnings.

File-Container / Codec Combinations Not all audio and video codes are permitted with all file container types. The following table indicates valid combinations of container type and audio/video encoder available with mtwriter:

Container Audio Encoder Video Encoder PCM AAC RGB H.264

WAV ∗ M4A ∗ M4V/MP4 ∗ ∗ ASF/WMV ∗ ∗ ∗1 ∗ AVI2 ∗ ∗ 1 Requires Windows 8.0 or later 2 Requires Windows 8.1

Default Codec Settings When either audio or video encoding information remains unspecified at initialization, mtwriter will revert to the following default settings for each container type:

Container Audio Encoder Video Encoder WAV PCM, 16-bit M4A AAC, 160 kbps M4V/MP4 AAC, 160 kbps H.264, quality 70 ASF/WMV PCM, 16-bit H.264, quality 70 AVI2 PCM, 16-bit RGB 1 Requires Windows 8.0 or later 2 Requires Windows 8.1

Initialization-Only Properties The following property may only be specified at the initialization of an mtwriter instance. 'SuppressWarnings' = flag that allows the user to suppress warnings that alert the user

to potential over-writing of existing files by mtwriter.

DEFAULT: 'false'

'UseFFMPEG' = ['false','true'] may be used to configure the toolset used by mtplayer to interact with AV files. Even though the mtwriter class does not interact with AV file natively, other Media Tools classes (mtplayer and mtreader) do. Since all of the Media Tools classes rely on the same underlying library of code, if any active classes set 'UseFFMPEG' to 'true', then all classes must set 'UseFFMPEG' to 'true'. Please see the help for either mtplayer or mtreader for details on FFMPEG. DEFAULT = 'false'.

Read-Write Properties

The following properties are available to the user both for reading and writing. 'AVFilename' = the name of the AV file associated with the writer. This is set to

FILENAME at initialization. If changed after initialization, the file being currently written will finalize the write and close and mtwriter will open the new file for writing. The new file MUST have the SAME container type as the original file. The audio and/or video codecs associated with the original file type will be preserved. The audio number of channels and sampling rate as well as the video pixel dimensions and frame rate may change as long as the new values are compatible with the codecs in use. To change the container type or codecs, it is necessary to create a new instance of mtwriter.

Read-Only Properties The following properties are available for read only. 'AVContainer' = the file container currently in use. Will be one of 'wav', 'm4a', 'm4v',

'asf', or'avi'.

'AudioFs' = the audio sampling rate in Hz. This value is inferred from the initial block of audio written to the file.

'AudioNumberChannels' = number of channels of file audio. This value is inferred from

the initial block of audio written to the file. 'AudioNumberSamples' = number of samples of audio that have been written to the file. 'Duration' = duration of content written to the file. Equal to the maximum of

DurationAudio and DurationVideo. 'DurationAudio' = duration of audio content written to the file. 'DurationVideo' = duration of video content written to the file.

'VideoFPS' = the video frame rate in frames per second. This value is inferred from the initial block of video written to the file.

'VideoNumberFrames' = number of video frames that have been written to the file. 'VideoPixelWidth' = width of video in pixels. This value is inferred from the initial block

of video written to the file. 'VideoPixelHeight' = height of video source in pixels. This value is inferred from the

initial block of video written to the file.

Methods appendAV(mtw, A), appendAV (mtw, V), appendAV (mtw, A, V), appendAV (mtw, V, A)

Append audio and/or video content to the mtwriter.

Audio content is specified as a structure A with two fields: o 'A.data' contains the channels of audio content stored column-wise. o 'A.Fs' contains the audio sampling rate.

Video content is specified as a structure V with two fields: o 'V.M' is an array of structures containing the video movie content. Specifically,

[V.M(1), V.M(2), ..., V.M(K)] specify K frames of video. Each V.M(k).cdata is a pix_height-by-pix_width-by-3 array of UINT8 values specifying the RGB values of the video frame. All K frames must be of equal dimensions. The R, G, and B values of each frame image occupy V.M(k).cdata(:,:,1), V.M(k).cdata(:,:,2), and V.M(k).cdata(:,:,3), respecively. The dimensions of the video content (in pixels) is inferred from the dimensions of V.M(k).

o 'V.fps' contains the video frame rate in frames per second.

If audio is being appended to a movie that already contains audio, then the number of channels and the audio sampling rate must match those of the existing audio. If video is being appended to a movie that already contains video, then the video pixel dimensions and the video frame rate must match those of the existing video. For files with both audio and video content, it is recommended to append audio and video in an approximately synchronous manner so that proper interleaving is maintained in an AV file. The mtwriter will automatically keep track of audio and video content. If the two streams become excessively out of sync, the user will be alerted and prevented from expanding this gap in synchronization.

close(mtw)

Finalize the mtwriter, flush all content to the file, and close the file. This function does not need to be called explicitly. The mtwriter will be closed automatically if it is deleted, cleared, or over-written.

equalizeAVDuration(mtw)

Equalize the durations of the audio and video content by either adding audio silence or duplicating the final frame. This functionality is useful when using mtwriter to concatenate multiple files. When concatenating, it is important that the content for each successive file start from exactly the same timestamp in both the audio and video streams - in order to ensure AV synchrony. Otherwise, the AV sync may drift as additional files are concatenated.

get(mtw,'Property')

Query value of mtwriter read-only 'Property'.

MediaToolsOutputDevices Query the system for a list of audio output devices associated with a particular audio API.

Syntax OUTPUT_DEV = MediaToolsOutputDevices(API);

Input Arguments API = the audio API to query. Valid values are: o 'MME' = legacy Windows audio o 'DirectSound' = legacy (but more recent than MME) Windows audio interface --

MATLAB generally uses this one o 'WASAPI' = most recent Windows audio interface (Vista and later) o 'ASIO' = the Steinberg Audio Stream Input/Output -- good for low-latency, high-speed

audio but requires special drivers to use

The native Windows APIs (MME, DirectSound, and WASAPI) generally present multi-channel audio devices as a series of stereo-pair devices. ASIO offers full multi-channel access. ASIO typically takes exclusive control of a particular device, which means that the same device cannot be used by multiple applications. The Media Tools for Matlab is regarded as a single application, and so multiple instances of mtplayer and mtmovie may use the same ASIO device.

Output Arguments OUTPUT_DEV = an array of output devices for the desired API. Each array element is a

structure describing a single device and has three fields. For device 'k': o 'OUTPUT_DEV(k).API' = the API of the device (same as the API input argument) o 'OUTPUT_DEV(k).DevName' = the name of the audio device o 'OUTPUT_DEV(k).nOutChannels' = the number of output channels available on the

particular device.

listActiveMediaToolsInstances Prints a list of all active instances of Media Tools for Matlab instances (mtplayer, mtmovie, mtreader, mtwriter).

Syntax listActiveMediaToolsInstances

clearAllMediaToolsInstances Clears and deletes all active instances of Media Tools for Matlab instances (mtplayer, mtmovie, mtreader, mtwriter).

Syntax clearAllMediaToolsInstances