linux sound programming
TRANSCRIPT
Contents
JAbout the Author xxi
About the Technical Reviewer xxiii
Chapter 1: Basic Concepts of Sound 1
Sampled Audio 1
Sample Rate 1
Sample Format 2
Frames 2
Pulse-Code Modulation 2
Overrun and Underrun 3
Latency 3
Jitter 4
Mixing 5
Conclusion 5
Chapter 2: User-Level Tools.... 7
Players 7
MPIayer 7
VLC 7
Totem 8
V
CONTENTS
Sound Tools 8
sox 8
FFmpeg/avconv 9
GStreamer 9
Audacity 9
Conclusion 9
Chapter 3: Sound Codecs and File Formats.......... 11
Overview 11
PCM 11
WAV 12
MP3 12
OggVorbis 12
WMA 13
Matroska 13
Conclusion 14
Chapter 4: Overview of Linux Sound Architecture 15
Resources 15
Components 15
Device Drivers 16
Sound Servers 17
Complexities 18
Conclusion 19
Chapter 5: ALSA 21
Resources 21
User Space Tools 21
alsamixer 21
alsactl 22
speaker-test 22
aplay/arecord 23
Identifying ALSA Cards 23
vi
CONTENTS
Device Names 24
ALSA Configuration Files 27
alsa-info 27
Applications Using ALSA 32
Programming ALSA 33
Hardware Device Information 34
PCM Device Information 37
Configuration Space Information 40
ALSA Initialization 46
Capture Audio to a File 46
Playback Audio from a File 49
Using Interrupts 52
Managing Latency 52
Playback of Captured Sound 52
Mixing Audio 58
Mixing Using dmix 58
Mixing Using PulseAudio 59
Simple Mixer API: Volume Control 59
Writing an ALSA Device Driver 60
Conclusion 60
Chapter 6: PulseAudio 61
Resources 61
Starting, Stopping, and Pausing PulseAudio 61
User Space Tools 62
paman 62
pavumeter 64
pavucontrol 64
Gnome Control Center (Sound) 66
parec/paplay/pacat 67
pactl/pacmd..... 68
Device Names 69
vii
CONTENTS
Loopback Module 69
PulseAudio andALSA 70
Programming with PulseAudio 70
Simple API 70
Play a File 71
Record to a File 73
Play from Source to Sink 76
Asynchronous API 78
List of Devices 79
Monitoring Ongoing Changes: New Sources and Sinks 84
Record a Stream 89
Play a File 93
Play a File Using I/O Callbacks 101
Controlling Latency 112
Play Microphone to Speaker 120
Setting the Volume on Devices 126
Listing Clients 128
Listing Client Sources and Sinks 133
Controlling the Volume of a Sink Client 138
Conclusion 141
Chapter 7: Jack 143
Resources 143
Starting Jack 143
User Tools 146
Applications Using Jack 146
mplayer 146
VLC 146
TiMidity 147
Jack-Supplied Programs 147
Other Jack Programs 147
viii
CONTENTS
Using a Different Sound Card 147
How Can I Use Multiple Sound Cards with Jack? 148
Mixing Audio 148
Writing Audio Applications with Jack 149
Compiling 150
Port Information 151
Copy Input to Output 153
Delaying Audio 157
Audacity with Jack 163
Play a Sine Wave 165
Saving Input to Disk 170
Interacting with ALSA Devices 177
Conclusion 177
Chapter 8: Session Management ...179
Resources 179
Session Management Issues 179
jack_connect 181
LASH 181
Jack Sessions 181
LADISH 184
Jack Session API 189
LADISH API 196
Conclusion 196
Chapter 9: Java Sound 197
Resources 197
Key Java Sound Classes 197
Information About Devices 198
Playing Audio from a File 199
ix
CONTENTS
Recording Audio to a File 202
Play Microphone to Speaker 206
Where Does JavaSound Get Its Devices From? 210
Conclusion 210
Chapter 10: GStreamer 211
Resources 211
Overview 211
Command-Line Processing 213
gst-inspect 213
gst-discoverer 215
gst-device-monitor 216
gst-play 216
gst-launch 217
C Programming 217
Writing Plug-ins 221
Conclusion 221
Chapter 11: libao 223
Resources 223
libao 223
Conclusion 225
Chapter 12: FFmpeg/Libav 227
Resources 227
The FFmpeg/Libav Controversy 227
FFmpeg Command-Line Tools 228
Programming 228
Decoding an MP3 File 229
Conclusion 234
X
CONTENTS
Chapter 13: OpenMAX IL 235
Resources 235
Quotes 236
OpenMAX IL Concepts 236
OpenMAX IL Components 237
Implementations 237
Raspberry Pi 237
Bellagio 238
LIM 238
Hardware-Supported Versions 239
Implementations of Components 239
Getting Information About an IL Component 244
Playing PCM Audio Files 260
State 260
Threads 260
Hungarian Notation in OpenMAX IL 262
Callbacks 262
Component Resources 262
Setting the Output Device 265
Main Loop 265
Complete Program 266
Conclusion 275
Chapter 14: LADSPA 277
Resources 277
User-Level Tools 277
The Type LADSPA_Descriptor 279
Loading a Plug-in 280
A Mono Amplifier Client 280
A Stereo Amplifer with GUI 286
xi
CONTENTS
The amp Program 291
Conclusion 292
Chapter 15: Displaying Video with Overlays Using Gtk and FFmpeg 293
FFmpeg 293
Basic Gtk 297
Versions of Gtk 299
Displaying the Video Using Gtk 300
Pixmaps 300
Pixbufs 300
X,Wayland, and Mir 300
Gtk 3.0 301
Gtk 2.0 301
Threads and Gtk 302
The Code 302
Overlaying an Image on Top of an Image 306
Alpha Channel 308
Using Cairo to Draw on an Image 309
Drawing Text Using Pango 312
Conclusion 315
Chapter 16: MIDI 317
Resources 317
Components of a MIDI System 317
Synthesizer 317
Sequencers 318
Other Components 318
MIDI Events 318
Program Change Events 318
Note Events 318
Meta Events 318
Conclusion 319
xii
CONTENTS
Chapter 17: User-Level Tools for MIDI 321
Resources 321
Sound Fonts 321
TiMidity 322
TiMidity as a Server 322
Setting TiMidity Output Device 323
TiMidity and Jack 323
GStreamer 323
fluidsynth 324
fluidsynth as a Server 324
Rosegarden 324
WildMIDI 324
Comparison 324
VLC 325
Conclusion 325
Chapter 18: MIDI Java Sound 327
Resources 327
Key Java Sound MIDI Classes 327
Device Information 328
Dumping a MIDI File 330
Playing a MIDI File 334
Changing the Soundbank 340
Changing Pitch and Speed 340
Using TiMidity Instead of the Default Gervill Synthesizer 342
Conclusion 342
Chapter 19: MIDI ALSA 343
Resources 343
ALSA Sequencer Clients 343
aconnect 344
xiii
CONTENTS
seqdemo 345
aplaymidi 346
Raw MIDI Ports 347
Raw MIDI Physical Devices 347
Raw MIDI Virtual Devices 347
Mapping MIDI Clients into MIDI Raw Space 348
Turning Off All Notes 349
Conclusion 349
Chapter 20: FluidSynth 351
Resources 351
Players 351
Play MIDI Files 352
Python 353
Conclusion 353
Chapter 21: TiMidity 355
TiMidity Design 355
Turning TiMidity into a Library 356
Managed Environments Hook 356
Building the Library 356
Library Entry Points 357
A Minimal Application 357
Playing a Background Video to a MIDI File 358
Building a New Interface 360
Shared Objects 360
Entry Point 360
ControlMode 361
Include Files 362
My Simple Interface 362
xiv
CONTENTS
Running My Simple Interface 367
Playing a Background Video to a MIDI File 367
Conclusion 368
Chapter 22: Overview of Karaoke Systems 369
Chapter 23: Karaoke User-Level Tools 371
Video CD Systems 371
CD+G Discs 372
MP3+G Files 372
Buying CD+G or MP3+G Files 373
Converting MP3+G to Video Files 373
MPEG-4 Files 373
Karaoke Machines 374
MIDI Players... 375
Finding MIDI Files 375
KAR File Format 376
PyKaraoke 377
kmid3 377
Microphone Inputs and Reverb Effects 379
Conclusion 379
Chapter 24: MP3+G 381
File Organization 384
Song Information 384
Song Table 386
Favorites 390
All Favorites 393
Swing Song Table 395
Playing Songs 401
XV
CONTENTS
VLC402
Playing Songs Across the Network 402
Conclusion 405
Chapter 25: Karaoke Applications Using Java Sound 407
Resources 408
KaraokePlayer 408
MidiPlayer 409
DisplayReceiver 413
MidiGUI 415
AttributedLyricPanel 420
PianoPanel 423
MelodyPanel 425
Sequencelnformation 430
PinYin 434
Karaoke Player with Sampling 435
Comments on Device Choices 441
Performance 441
Conclusion 441
Chapter 26: Subtitles and Closed Captions 443
Resources 443
Subtitle Formats 443
MPIayer 443
VLC 444
Gnome Subtitles 444
Substation Alpha 445
Karaoke Effects in ASS Files 446
Multiline Karaoke 449
libass 451
Converting KAR Files to MKV Files with ASS Subtitles 451
xvi
CONTENTS
HTML5 Subtitles 463
Conclusion 464
Chapter 27: Karaoke FluidSynth 465
Resources 465
Players 465
Play MIDI Files 465
Extending FluidSynth with Callbacks 467
Displaying and Coloring Text with Gtk 470
Playing a Background Video with Gtk 482
Conclusion 497
Chapter 28: TiMidity and Karaoke 499
TiMidity and Jack 500
TiMidity Interface 502
Getting the List of Lyrics 504
TiMidity Options 505
Playing Lyrics Using Pango + Cairo + Xlib 506
Playing a Background Video with Gtk 512
Background Video with TiMidity as Library 519
Background Video with TiMidity as Front End 521
Adding Microphone Input 521
Conclusion 521
Chapter 29: Jack and Karaoke 523
Using Jack Rack for Effects 523
Playing MIDI 526
TiMidity Plus Jack Rack 527
Customizing TiMidity Build 528
Playing MP3+G with Jack Rack Pitch Shifting 528
Conclusion 532
xvii
CONTENTS
Chapter 30: Streaming Audio 533
HTTP 533
HTTP Servers 533
HTTP Clients 533
Streaming vs. Downloading 534
HTML5 534
DLNA 535
Icecast 535
Flumotion 535
Conclusion 535
Chapter 31: Raspberry Pi 537
Resources 537
The Basics 537
Hardware 537
Alternative Single-Board Computers 538
Distros 539
No Sound 539
ALSA 539
Sampled Audio Players 541
MPIayer 541
VLC 541
alsaplayer 541
omxplayer 541
Is It X Using the CPU? 541
Sampled Audio Capture 541
ALSA 541
MIDI Players 542
TiMidity 542
pykaraoke 542
FluidSynth/qsynth 542
xviii
CONTENTS
Java Sound 543
PulseAudio 544
Java MIDI 545
OpenMAX 545
Conclusion 545
aChapter 32: Conclusion 547
Where Did I Start? 547
Where Did I Get To? 547
How Did I Get There? 547
Appendix A: Decoding the DKD Files on the Sonken Karaoke DVD 549
Introduction 549
Format Shifting 550
Files on the DVD 550
BACK01.MPG 551
DTSMUSOO.DKD to DTSMUS07.DKD 551
DTSMUS10.DKD 551
DTSMUS20.DKD 551
Decoding DTSMUS20.DKD 551
Song Information 551
Beginning/End of Data 553
Chinese Songs 553
Other Languages 554
Programs 554
The Data Files 568
General 568
My Route into This 569
The Superblock 570
Song Start Tables 571
Locating Song Entry from Song Number 571
xix