audio engine programming
DESCRIPTION
Audio Engine Programming. Lecture 4 Introduction to FMOD programmers API. 3D Sound. Sounds have position and velocity There is a listener component Relationship between the two Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities) - PowerPoint PPT PresentationTRANSCRIPT
Audio Engine Programming
Lecture 4
Introduction to FMOD programmers API
3D Sound
Sounds have position and velocity There is a listener component Relationship between the two
Attenuation (with distance) Occlusion (low-pass filter) Doppler (relative velocities)
Lots of “psycho-acoustic” options
Fmod audio development
There are three primary components: The Sound Designer The Sandbox The API
We’ll use all three Start with Sound Designer
Data Driven vs. Low Level API Low-Level API
Everything is done using programming The programmer has complete control
The benefits are lower memory overhead
Continued…
Data-Driven API Takes assets from FMOD Designer
Sound Designer has control over the audio Integrates them into the code using the
EventSystem object
Key Features
The data-driven approach of the EventSystem: Support’s event logic
Allows the Sound Designer to fine-tune or ‘tweak’ the final mix without the need for Programmer intervention
Creates a logical separate of concerns for the Programmer and Sound Designer
Overview of event model
FMOD Low Level
FMOD EventSystem
FMOD FMOD EXEX
FEVevent data
FSBaudio data
Assets from the sound designer
The build process creates the following assets: .FEV file - contains the event meta data .FSB file(s) - contains the audio data The ‘Programmer report ‘- a human readable list of
event names, sound definitions [Optional] A ‘header’ file - A source code header file [Optional] Cache files - Files used to speed up subsequent
builds (These files are not required by the Programmer)
Linking the library
Include the EventSystem header file #include "fmod_event.hpp“
Declare a pointer to an EventSystem FMOD::EventSystem *eventsystem;
Create the object (allocate memory) FMOD::EventSystem_Create(&eventsystem);
Initialize the object eventsystem->init(64, FMOD_INIT_NORMAL, 0,
FMOD_EVENT_INIT_NORMAL);
Loading files
Set the media path (location of files) eventsystem->setMediaPath((char *)MEDIA_PATH);
Load the .FEV file eventsystem->load("examples.fev", 0, 0);
Load an event group eventsystem->getGroup("examples/examples", FMOD_EVENT_DEFAULT,
&eventgroup);
Load an individual event eventgroup->getEvent(“car", FMOD_EVENT_DEFAULT, &car);
Loading Audio
The required .FSB file(s) are listed in .FEV file Both the .FEV and .FSB files should be located in the
directory specified with setMediaPath() The location of audio within the .FSB files is
stored in the .FEV FMOD will load the audio data from the .FSB
files as required for an event, automatically
Loading Options
FMOD_EVENT_DEFAULT Specifies default loading behaviour Event data for the whole group is NOT cached Loading will block any other task until complete.
FMOD_EVENT_NONBLOCKING FMOD will use a thread to load the data Use Event::getState to find out when loading is
complete.
Continued...
FMOD_EVENT_INFOONLY Does not allocate instances or load data Creates a ‘handle’ that allows the programmer to
get information from the event
EVENT Playback
Each events have a number of methods and attributes
To start playback of an event: example->start();
To stop an event: Example->stop();
Updating
Updating the event system should be called once per 'game' tick (or once per frame) eventsystem->update();
This allows the FMOD Engine to refresh data such as: the position of 3D sounds changes to volume, pitch, effects as well as internal mechanisms
Demonstration
Simple Event Demonstrates the necessary code to start
playback of an event
PROVIDING Parameter DATA
Get the parameter called ‘test’ example->getParameter(“test", &test); test will be a EventParameter object
To get the range of the parameter test->getRange(&rangemin, &rangemax);
To set the value of the parameter test->setValue(value);
Demonstration Parameter
Demonstrates the necessary code to provide parameter data to an event
PROVIDING a KEY-OFF If an event parameter is currently sustaining
on a ‘sustain point’, triggering a keyoff will release it and allow the parameter to continue test->keyoff()
Demonstration
Keyoff Demonstrating the necessary code to leave a
sustain point
Selecting sounds at run-time
The sound to be played can be selected at runtime This is required for situations such as ‘colour
commentary’ in sport games or other situations that require dynamic dialogue
The are two methods: Programmer Sounds Programmer Selected
Demonstration
Programmer Sound When a sound definition containing a ‘ProgrammerSound’ is to
be played, a callback is made The code must then provide a sound object at runtime just
before it's played
Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSound’ method
Demonstration
Programmer Selected When a sound definition with the playmode
‘ProgrammerSelected’ is to be played, a callback is made The code must then provide the index number at runtime just
before it's played The index number identifies which file within the sound definition
is to be played
Demonstrates the necessary code to select a sound to play at runtime, using the ‘ProgrammerSelected’ method
Closing the Event System
Unload event data eventsystem->unload();
Release audio data in memory fsb->release();
Release memory and close eventsystem memory eventsystem->release();
Setting up a Project
Take the advice of the video tutorials
Have a separate folder Copy sounds into a “sounds” directory
Keeps the safe Can have sub-directories
Create an “output” directory
Designer Interface
Can delete this if you want, but need to have atleast one group
Events
Used to define sound properties for an event Can be comprised of
One sound Several sounds randomly chosen Monophonic vs Polyphonic Can randomize pitch and attenuation
Events can be Simple Multi-track
A Simple Event
Granular Sounds
These sounds aren’t looping
Allows for a sounds to occur between time ranges
Allows for polyphony
Event Options
Can vary attenuation (dropoff with distance)
Can vary pitch Creates a sound
considerably smaller than a “soundtrack”
Plays forever and is always random!
Sound Definitions
Required for multi-track events
Multi-track Events
Comprised of sound defs in layers
Multi-track Events
Careful when adding sounds Choose sound def Ambient noise is looping Other sounds are granular,
so choose “Oneshot” Have parameters Not based on time!
Multi-track Events
Can cross-fade and set fade out time
Effects
Parameter Properties
Can define range and velocity (to simulate time)
Engine Designer
fmod can specifically work with engine sounds Need for Speed 2
Based on the “load” of the engine Right-click on a
sound instance->properties Auto-pitch
Window->fmod Engine Designer
The Build Process
Know which platform you’re targeting Changes are applied only to that platform
Project->Clean, Project->Build
Interactive Music
Comprised of several short segments of music Intro music Darkening or discovery Fighting/intense fighting Release
Before you begin, you must know Tempo of music (beats per minute – bpm) Beats per measure (time signature)
Cues
Themes and Auditioning
Transitioning via Parameters
Other Things
Supports deployment of multiple languages Can deploy different builds based on
Platform Language
A Look at the API
Basic API Designer API
Built on basic API Can read .fev files
Written for C/C++ #include <fmod.h>// C #include <fmod.hpp> // C++
Read the documentation fmodex.chm
API Parts A System is the fmod engine A Sound is the raw resource
2D uses FMOD_2D 3D uses FMOD_3D
result = system->createSound(“music.wav", FMOD_2D, 0, &sound);
API Parts
A Channel is an instance of a Sound Each time you play a sound, you get a new
channel Channels can start out paused You can set the
Volume (0.0f – 1.0f) Pan (-1.0f – 1.0f) Frequency (in Hz as a float)
Always use FMOD_CHANNEL_FREE to pick for you
#include <iostream>#include <fmod.hpp>#include <fmod_errors.h>
using namespace std;using namespace FMOD;
void main () {FMOD_RESULT result;System* system;// Create the fmod system. We only need one of theseresult = System_Create(&system);// Specify a max number of simultaneous channelsresult = system->init(100, FMOD_INIT_NORMAL, 0);
Sound* sound;// Decompress the entire mp3 into 16-bit PCM in memoryresult = system->createSound("winning.mp3", FMOD_DEFAULT, 0, &sound);
if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;
}
Channel* channel; // Used for setting volume, pan, pausing...// You have the option of passing that channel as the last parameterresult = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency (44100.0f);
if (result != FMOD_OK) {
cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}while (true) {
cout << "1"; system->update(); // Now required in fmod}
}
Virtual Voices Supports “virtual voices” when hardware is
limited Set from System::init() System::getCPUUsage() Try to keep < 1000
May need to set the priority of a channel if it’s important
channel->isVirtual()
3D Sound in fmod
Sound attenuation Logarithmic Set the mindistance of a sound channel to start
attenuation Bee = 0.1f Jet = 50.0f
Leave max distance alone (default is 10,000) Set 3D settings with System::set3DSettings()
Doppler Distance factor (in cm, m, feet) Rolloff scale (attenuation models)
3D Sound
fmod uses a left-handed coordinate systemresult=system->init(100,
(FMOD_MODE)(FMOD_INIT_3D_RIGHTHANDED|FMOD_3D),0);
With velocity, you must pass it as metres per second
velx = (posx-lastposx) * 1000 / timedelta;
vel = 0.1 * 1000 / 16.67 = 6 meters per second
vel = 0.2 * 1000 / 33.33 = 6 meters per second
void main () {FMOD_RESULT result;FMOD_VECTOR soundPos, soundVel; FMOD_VECTOR listenerPos, listenerVel, listenerForward, listenerUp;
System* system;result = System_Create(&system);result = system->init(100, (FMOD_MODE)(FMOD_3D), 0);
int numHC = 0;result = system->getHardwareChannels(&numHC);cout << "Hardware channels: " << numHC << endl;
Sound* sound;result = system->createSound(“train.mp3", FMOD_3D, 0, &sound);
if (result != FMOD_OK) {cout << "Couldn't open it! " << FMOD_ErrorString(result) << endl;
}
Channel* channel;result = system->playSound(FMOD_CHANNEL_FREE, sound, false, &channel);channel->setFrequency(44100.0f); if (result != FMOD_OK) {
cout << "Couldn't play it! " << FMOD_ErrorString(result) << endl;}float dsp, stream, geometry, update, total;soundPos.y = soundVel.x = soundVel.y = 0.0f;soundPos.z = -100.0f; soundPos.x = 5.0f;soundVel.z = 6.0f;channel->set3DMinMaxDistance(10, 10000);listenerPos.x = listenerPos.y = listenerPos.z = 0.0f;listenerVel.x = listenerVel.y = listenerVel.z = 0.0f;listenerForward.x = listenerForward.y = listenerUp.x = listenerUp.z = 0.0f;listenerForward.z = listenerUp.y = 1.0f;system->set3DListenerAttributes(0, &listenerPos, &listenerVel, &listenerForward, &listenerUp);
while (true) {system->update(); system->getCPUUsage(&dsp, &stream, &geometry, &update, &total);cout << total << endl; soundPos.z+=0.01f;channel->set3DAttributes(&soundPos, &soundVel);
}}
Interfacing with Sound Designer
You should have created .fev file (Designer file – event data) .fsb file (raw audio data) .txt file (describes the events and parameters)
May want to start by creating a helper function
void checkErrors(FMOD_RESULT result) { if (result != FMOD_OK) { cout << "fmod error: " << FMOD_ErrorString(result) << endl; exit(1); }}
Imagine That This is our Project
#include <iostream>#include <fmod.hpp>#include <fmod_event.hpp>#include <fmod_errors.h>using namespace std;using namespace FMOD;
void main () { EventSystem* eventSystem = NULL; EventGroup* eventGroup = NULL; Event* myEvent = NULL; EventParameter* eventPar;
FMOD_RESULT result = EventSystem_Create(&eventSystem); checkErrors(result); result = eventSystem->init(64, FMOD_INIT_NORMAL, 0, FMOD_EVENT_INIT_NORMAL); eventSystem->setMediaPath("..//"); result = eventSystem->load("fmodTest.fev", 0, 0); result = eventSystem->getGroup("fmodTest/beeps", false, &eventGroup); result = eventGroup->getEvent("PossessedComputer", FMOD_EVENT_DEFAULT, &myEvent); result = myEvent->getParameter("proximityToComputer", &eventPar); myEvent->start();
float dir = 0.0001f; float currentParVal = -1.0f; eventPar->setValue(0.1f); while (true){
cout << currentParVal << endl;eventPar->getValue(¤tParVal);currentParVal+=dir;eventPar->setValue(currentParVal);if ((currentParVal >= 1.0)||(currentParVal <= 0.0)) {
dir = -dir;}
}}
Interactive Music
void main () {EventSystem* eventSystem = NULL;MusicSystem* musicSystem = NULL;MusicPrompt* introPrompt, *fightPrompt, *fight2Prompt, *releasePrompt;FMOD_MUSIC_ITERATOR cueIter;FMOD_MUSIC_ITERATOR paramIter;FMOD_MUSIC_ID intensityParID = -1.0f;
FMOD_RESULT result = EventSystem_Create(&eventSystem);result = eventSystem->init(64, FMOD_INIT_NORMAL, 0,
FMOD_EVENT_INIT_NORMAL);result = eventSystem->load("acid_test.fev", 0, 0);result = eventSystem->getMusicSystem(&musicSystem);musicSystem->setVolume(1.0f);result = musicSystem->getCues(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fightPrompt);result = musicSystem->getNextCue(&cueIter);cout << "Iter ID: " << cueIter.value->id << endl;result = musicSystem->prepareCue(cueIter.value->id, &fight2Prompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &introPrompt);result = musicSystem->getNextCue(&cueIter);result = musicSystem->prepareCue(cueIter.value->id, &releasePrompt);
musicSystem->getParameters(¶mIter);intensityParID = paramIter.value->id;cout << "paramIter.value->name is " << paramIter.value->name << endl;musicSystem->setParameterValue(intensityParID, 7.5f);result = introPrompt->begin();
while (true){eventSystem->update();
}}
Other Things You Should Look Into…
The 3D Reverb API Asynchronously loading data (by default) Memory management for non-PC platforms
FMOD::Memory_Initialize() // fix size of fmod Using compressed samples
Built-in DSP
Configuration Notes
You’ll be working with Header files
C/C++ -> General->Additional Include Directories Lib files
Linker->General->Additional Library Directories Linker->General->Additional Dependencies
DLLs (which you should put into Windows/System32)
There’s also a FMOD.NET site…
Configuration Notes
You have the option of 64-bit and 32-bit libraries You might have to create a new platform