mul$mediatechniques* inandroid · mediaapi comp 355 (muppala) multimedia 3 audio/video* playback*...
TRANSCRIPT
![Page 1: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/1.jpg)
Mul$media Techniques in Android
Some of the informa$on in this sec$on is adapted from WiseAndroid.com
![Page 2: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/2.jpg)
Mul$media Support
• Android provides comprehensive mul$media func$onality: – Audio: all standard formats including MP3, Ogg, Midi, … – Video: MPEG-‐4, H.263, H.264, – Images: PNG (preferred), JPEG, GIF (not recommended)
• Several different media formats and codecs are supported • You can play audio or video from media files stored in the
applica$on's resources (raw resources), from standalone files in the filesystem, or from a data stream arriving over a network connec$on.
COMP 355 (Muppala) Multimedia 2
![Page 3: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/3.jpg)
Media API
COMP 355 (Muppala) Multimedia 3
Audio/Video
Playback Record
MediaPlayer SoundPool MediaRecorder
![Page 4: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/4.jpg)
Media Playback and Recording
• Standard method to manipulate audio/video: – Media playback supported through the MediaPlayer and SoundPool
classes – Media recording supported through the MediaRecorder class – Creates its own thread for processing – Requires audio as file or stream based data
• Direct access to raw audio: – AudioTrack and AudioRecorder classes – Useful to manipulate audio in memory, manipula$ng the audio buffer,
or any other usage not requiring file or stream data – Does not create its own thread
COMP 355 (Muppala) Multimedia 4
![Page 5: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/5.jpg)
Media Playback
• MediaPlayer is designed for longer sound files or streams (larger files) – The files will be loaded from disk each $me create is called, this will
save on memory space but introduce a small delay (not really no$ceable)
• SoundPool – is designed for short clips which can be kept in memory decompressed
for quick access – suited for sound effects in apps or games – loads lots of “medium” sized sounds into the memory and you may
exceed your limit (16Mb) and get an OutOfMemoryExcep$on.
COMP 355 (Muppala) Multimedia 5
![Page 6: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/6.jpg)
MediaPlayer Methods
• Crea$ng a media player: MediaPlayer player = new MediaPlayer();
• Specifying the source of the media: – If file in the resource directory raw:
player = MediaPlayer.create(context, R.raw.music_file); – If file or stream:
player.setDataSource(path); player.prepare();
• Start playback player.start();
• Pause playback player.pause();
• Stop playback player.stop(); player.release();
COMP 355 (Muppala) Multimedia 6
![Page 7: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/7.jpg)
MediaPlayer State Diagram
COMP 355 (Muppala) Multimedia 7
![Page 8: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/8.jpg)
SoundPool Methods
• Crea$ng a media player: SoundPool soundPool= new SoundPool(2, AudioManager.STREAM_MUSIC, 100);
• Specifying the source of the media: – If file in the resource directory raw:
SoundPoolMap soundPoolMap = new HashMap<Integer, Integer>(); soundPoolMap.put(streamID, soundPool.load(this.getBaseContext(), R.raw.birdsound, 1));
– If file or stream: soundPool.load(String path, int priority);
• Start playback – soundPool.play(streamID, streamVolume, streamVolume, 1, 0, 1f);
• Pause playback – soundPool.pause(streamID);
• Stop playback – soundPool.stop(streamID); – soundPool.unload(int soundID);
COMP 355 (Muppala) Multimedia 8
![Page 9: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/9.jpg)
Video Playback and Recording • Video playback and recording also uses the MediaPlayer and
MediaRecorder classes with similar steps as in audio • Use VideoView widget in the ac$vity UI screen to host video:
<VideoView android:id="@+id/video" android:layout_width="fill_parent" android:layout_height="fill_parent"> </VideoView>
• Get a handle to the video view in your code: • videoView = (VideoView) this.findViewById(R.id.video);
• Set the path to the video file: • videoView.setVideoURI(Uri.parse(“\sdcard\video.mp4”));
• Set the focus: • videoView.setFocus();
• You can use a MediaController to control the video • MediaController mc = new MediaController(this); • videoView.setMediaController(mc);
COMP 355 (Muppala) Multimedia 9
![Page 10: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/10.jpg)
MediaRecorder Methods • Create a new instance of Media recorder:
– MediaRecorder recorder = new MediaRecorder(); • Set the audio source:
– recorder.setAudioSource(MediaRecorder.AudioSource.MIC); • Set output file format:
– recorder.setOutputFileFormat(MediaRecorder.OutputFormat.THREE_GPP); • Set output file name:
– recorder.setOutputFile(PATH); • Set the audio encoder:
– recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB); • Prepare the media recorder:
– recorder.prepare(); • Start capture
– recorder.start(); • Stop capture
– recorder.stop(); • Finish recording
– recorder.release();
COMP 355 (Muppala) Multimedia 10
![Page 11: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/11.jpg)
MediaPlayer and MediaRecorder
• Rich API designed for long playing media streams, such as voice audio recordings, music and videos – seeking opera$ons supported – Source buffering
• Heavyweight resource-‐wise • Slow to ini$alize • Not suitable for low-‐latency scenarios like playing short audio
samples for sound effects, such as in games. • Designed for situa$ons with no more than one or two
MediaPlayers working at the same $me
COMP 355 (Muppala) Multimedia 11
![Page 12: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/12.jpg)
SoundPool Class
• The SoundPool class manages and plays audio resources for applica$ons.
• A SoundPool is a collec$on of samples that can be loaded into memory from a resource inside the APK or from a file in the file system
• The SoundPool library uses the MediaPlayer service to decode the audio into a raw 16-‐bit PCM mono or stereo stream
• This allows applica$ons to ship with compressed streams without having to suffer the CPU load and latency of decompressing during playback
• Typical usage scenario includes games with sound where several sounds are used in a level and may have to be played with overlap some$mes
COMP 355 (Muppala) Multimedia 12
![Page 13: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/13.jpg)
SoundPool Class Features
• Senng the maximum number of sounds to play at the same $me
• Priori$zing the sounds so that the low-‐priority ones will be dropped when the maximum threshold is reached
• Pausing and stopping sounds before they finish playing • Looping sounds • Changing the playback rate (effec$vely, the pitch of each
sound) • Senng stereo volume (separate for lep and right channels)
COMP 355 (Muppala) Multimedia 13
![Page 14: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/14.jpg)
AudioTrack Class
• The AudioTrack class manages and plays a single audio resource for Java applica$ons
• It allows to stream PCM audio buffers to the audio hardware for playback. This is achieved by "pushing" the data to the AudioTrack object one of the write(byte[], int, int) and write(short[], int, int) methods.
• An AudioTrack instance can operate under two modes: sta$c or streaming.
• In Streaming mode, the applica$on writes a con$nuous stream of data to the AudioTrack, using one of the write() methods.
• The streaming mode is most useful when playing blocks of audio data that for instance are: – too big to fit in memory because of the dura$on of the sound to play, – too big to fit in memory because of the characteris$cs of the audio data (high
sampling rate, bits per sample ...) – received or generated while previously queued audio is playing.
COMP 355 (Muppala) Multimedia 14
![Page 15: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/15.jpg)
AudioTrack Class
• The sta$c mode is to be chosen when dealing with short sounds that fit in memory and that need to be played with the smallest latency possible
• AudioTrack instances in sta$c mode can play the sound without the need to transfer the audio data from Java to na$ve layer each $me the sound is to be played
• The sta$c mode will therefore be preferred for UI and game sounds that are played open, and with the smallest overhead possible.
• Upon crea$on, an AudioTrack object ini$alizes its associated audio buffer • The size of this buffer, specified during the construc$on, determines how
long an AudioTrack can play before running out of data • For an AudioTrack using the sta$c mode, this size is the maximum size of
the sound that can be played from it • For the streaming mode, data will be writen to the hardware in chunks of
sizes inferior to the total buffer size.
COMP 355 (Muppala) Multimedia 15
![Page 16: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/16.jpg)
AudioTrack Class
• AudioTrack class allows us the flexibility of: – Decoding audio from any format that is not supported by the plavorm – Modifying or enhancing audio on the fly (such as applying distor$on,
vocoder, reverb effects – but you have to code them yourself!) – Loading audio from custom sources to play it on the fly (but you have
to deal with buffering and stability yourself – be sure not to develop another MediaPlayer!)
• The price is that you are on your own regarding buffering, latency and correctness of the sound you produce with AudioTrack.
COMP 355 (Muppala) Multimedia 16
![Page 17: Mul$mediaTechniques* inAndroid · MediaAPI COMP 355 (Muppala) Multimedia 3 Audio/Video* Playback* Record* MediaPlayer* SoundPool* MediaRecorder*](https://reader030.vdocument.in/reader030/viewer/2022041007/5ead703b4279de4e1b090d15/html5/thumbnails/17.jpg)
Android Audio Comparison Audio Requirements Audio Class Choice
Require low latency, such as in games or sound effects
SoundPool/ AudioTrack (for more flexibility)
Need to play video that has an audio track MediaPlayer
Play a set of short sounds many $mes SoundPool
Stream audio from an external source, e.g. HTTP or TCP stream
MediaPlayer/ AudioTrack (if MediaPlayer not suitable)
Generate audio from scratch, such as by using math formulas / frequency modula$on
AudioTrack
Play background music MediaPlayer
Modify the audio on the fly AudioTrack
COMP 355 (Muppala) Multimedia 17