![Page 1: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/1.jpg)
These are confidential sessions—please refrain from streaming, blogging, or taking pictures
Session 602
What’s New in Core Audio for iOS
Tony GuettaManager, Core Audio System Software
![Page 2: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/2.jpg)
Introduction
•Overview of new audio features in iOS 7• Focus on one exciting new technology
![Page 3: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/3.jpg)
Audio Input Selection
• Select among available audio inputs• Choose which microphone on multi-mic devices• Set microphone polar pattern
■ Achieves directivity through beam forming processing■ e.g., cardioid, subcardioid
• See <AVFoundation/AVAudioSession.h>
![Page 4: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/4.jpg)
Multichannel Audio Enhancements
•Discover maximum number of input and output channels• Set preferred number of input and output channels•Get audio channel labels• See <AVFoundation/AVAudioSession.h>
![Page 5: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/5.jpg)
Open AL Extensions
• Per-source spatialization rendering quality■ Improved high-quality rendering algorithm■ Render to multichannel output hardware
•Output capture• See <OpenAL/oalMacOSX_OALExtensions.h>
![Page 6: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/6.jpg)
Audio Queue Time-Pitch Capabilities
• Set playback rate•Adjust playback pitch• See <AudioToolbox/AudioQueue.h>
![Page 7: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/7.jpg)
Audio Recording Permission
• Recording now requires user approval•One-time approval remembered for each application• Changeable in Settings• Silence until permission is granted• Triggered by use of AVAudioSession categories that enable recording•API for application control{
... [[AVAudioSession sharedInstance] requestRecordPermission];
...}
![Page 8: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/8.jpg)
Deprecated Audio Session C API
•Use AVAudioSession• See <AVFoundation/AVAudioSession.h>
![Page 9: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/9.jpg)
SummaryNew in iOS 7
•Audio input selection•Multichannel audio enhancements•Open AL extensions•Audio Queue time-pitch capabilities•Audio recording permission•Audio Session C API officially deprecated—Use AVAudioSession
![Page 10: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/10.jpg)
SummaryNew in iOS 7
•Audio input selection•Multichannel audio enhancements•Open AL extensions•Audio Queue time-pitch capabilities•Audio recording permission•Audio Session C API officially deprecated—Use AVAudioSession• Inter-app audio
![Page 11: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/11.jpg)
Inter-App Audio
• Stream audio between apps in real-time• Built on familiar APIs
![Page 12: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/12.jpg)
Demo
Alec LittleGarageBand
![Page 13: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/13.jpg)
Inter-App Audio in Detail
Doug Wyatt Core Audio Plumber
![Page 14: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/14.jpg)
Host App Node App
API Overview
![Page 15: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/15.jpg)
Host App Node App
RegisterDiscover
System
API Overview
![Page 16: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/16.jpg)
Host App Node App
RegisterDiscover
System ConnectConnect
API Overview
![Page 17: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/17.jpg)
Host App Node App
RegisterDiscover
System ConnectConnect
Audio
API Overview
![Page 18: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/18.jpg)
Host App Node App
RegisterDiscover
System ConnectConnect
Audio
MIDI
API Overview
![Page 19: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/19.jpg)
Host App Node App
RegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
API Overview
![Page 20: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/20.jpg)
Host App Node App
RegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
API Overview
![Page 21: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/21.jpg)
Host appIntroduction
AURemoteIO
Host App Engine
![Page 22: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/22.jpg)
Host appIntroduction
AURemoteIO
Host App Engine
Node AppNode Audio Unit
![Page 23: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/23.jpg)
Node appIntroduction
AURemoteIO
Node App Engine
![Page 24: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/24.jpg)
Node appIntroduction
AURemoteIO
Node App Engine
Host App
![Page 25: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/25.jpg)
Introduction
• Extensions to AudioUnit.framework API•Host sees node as AudioUnit•Node’s I/O unit redirected to host
![Page 26: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/26.jpg)
New AudioUnit typesIntroduction
Audio Component Type Input (from Host) Output
kAudioUnitType_RemoteGenerator - audio
kAudioUnitType_RemoteInstrument MIDI audio
kAudioUnitType_RemoteEffect audio audio
kAudioUnitType_RemoteMusicEffect audio and MIDI audio
![Page 27: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/27.jpg)
Generator or instrumentOne Node App, Multiple Components
MIDIHost App Node App
Audio
![Page 28: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/28.jpg)
Generator or effectOne Node App, Multiple Components
Host App Node App
Audio
Generator
![Page 29: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/29.jpg)
Generator or effectOne Node App, Multiple Components
Host App Node App
Audio
Audio
Effect
![Page 30: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/30.jpg)
RequirementsIntroduction
•Available on most iOS 7-compatible devices■ (Except iPhone 4: API’s fail quietly)
• “inter-app-audio” entitlement (all)• “audio” in UIBackgroundModes (hosts, nodes that access mic)•AVAudioSessionCategoryOptionMixWithOthers (nodes)
![Page 31: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/31.jpg)
Registering a Node AppRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 32: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/32.jpg)
Registering a Node App
• Register via Info.plist “AudioComponents”■ Makes app launchable
•AudioOutputUnitPublish■ “Checks in” the registration
![Page 33: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/33.jpg)
AudioComponents entry in Info.plistRegistering a Node App
<key>AudioComponents</key><array>! <dict>! ! <key>type</key>! ! <string>aurg</string>! ! <key>subtype</key>! ! <string>ACgn</string>! ! <key>manufacturer</key>! ! <string>ACME</string>! ! <key>name</key>! ! <string>Acme: SineGenerator</string>! ! <key>version</key>! ! <integer>1</integer>! </dict></array>
![Page 34: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/34.jpg)
Create and publish the I/O unitRegistering a Node App
// create the AURemoteIO I/O unit
AudioComponentDescription ioUnitDesc = { kAudioUnitType_Output, kAudioUnitSubType_RemoteIO, kAudioUnitManufacturer_Apple, 0, 0 };
AudioComponent comp = AudioComponentFindNext(NULL, &ioUnitDesc);!
err = AudioComponentInstanceNew(comp, &_ioUnit);
// publish the AURemoteIO
AudioComponentDescription generatorDesc = { kAudioUnitType_RemoteGenerator,‘ACgn’, ‘ACME’, 0, 0 };
CFStringRef name = CFSTR(“Acme: SineGenerator”);
const UInt32 version = 1;
err = AudioOutputUnitPublish(&generatorDesc, name, version, _ioUnit);
![Page 35: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/35.jpg)
Create and publish the I/O unitRegistering a Node App
// create the AURemoteIO I/O unit
AudioComponentDescription ioUnitDesc = { kAudioUnitType_Output, kAudioUnitSubType_RemoteIO, kAudioUnitManufacturer_Apple, 0, 0 };
AudioComponent comp = AudioComponentFindNext(NULL, &ioUnitDesc);!
err = AudioComponentInstanceNew(comp, &_ioUnit);
// publish the AURemoteIO
AudioComponentDescription generatorDesc = { kAudioUnitType_RemoteGenerator,‘ACgn’, ‘ACME’, 0, 0 };
CFStringRef name = CFSTR(“Acme: SineGenerator”);
const UInt32 version = 1;
err = AudioOutputUnitPublish(&generatorDesc, name, version, _ioUnit);
![Page 36: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/36.jpg)
Create and publish the I/O unitRegistering a Node App
// create the AURemoteIO I/O unit
AudioComponentDescription ioUnitDesc = { kAudioUnitType_Output, kAudioUnitSubType_RemoteIO, kAudioUnitManufacturer_Apple, 0, 0 };
AudioComponent comp = AudioComponentFindNext(NULL, &ioUnitDesc);!
err = AudioComponentInstanceNew(comp, &_ioUnit);
// publish the AURemoteIO
AudioComponentDescription generatorDesc = { kAudioUnitType_RemoteGenerator,‘ACgn’, ‘ACME’, 0, 0 };
CFStringRef name = CFSTR(“Acme: SineGenerator”);
const UInt32 version = 1;
err = AudioOutputUnitPublish(&generatorDesc, name, version, _ioUnit);
![Page 37: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/37.jpg)
Publishing the I/O unitRegistering a Node App
•Must publish on launch• Component descriptions, names, and versions must match• Component name should be “Manufacturer: App name”
![Page 38: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/38.jpg)
Node App Discovery (for Hosts)RegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 39: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/39.jpg)
For hostsNode App Discovery
AudioComponentDescription searchDesc = { 0, 0, 0, 0, 0 }, foundDesc;AudioComponent comp = NULL;
while (true) {comp = AudioComponentFindNext(comp, &searchDesc);!if (comp == NULL) break;
if (AudioComponentGetDescription(comp, &foundDesc) != noErr) continue;
switch (foundDesc.componentType) {case kAudioUnitType_RemoteEffect:case kAudioUnitType_RemoteGenerator:case kAudioUnitType_RemoteInstrument:case kAudioUnitType_RemoteMusicEffect:// found a node...break;
}}
![Page 40: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/40.jpg)
For hostsNode App Discovery
AudioComponentDescription searchDesc = { 0, 0, 0, 0, 0 }, foundDesc;AudioComponent comp = NULL;
while (true) {comp = AudioComponentFindNext(comp, &searchDesc);!if (comp == NULL) break;
if (AudioComponentGetDescription(comp, &foundDesc) != noErr) continue;
switch (foundDesc.componentType) {case kAudioUnitType_RemoteEffect:case kAudioUnitType_RemoteGenerator:case kAudioUnitType_RemoteInstrument:case kAudioUnitType_RemoteMusicEffect:// found a node...break;
}}
![Page 41: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/41.jpg)
For hostsNode App Discovery
AudioComponentDescription searchDesc = { 0, 0, 0, 0, 0 }, foundDesc;AudioComponent comp = NULL;
while (true) {comp = AudioComponentFindNext(comp, &searchDesc);!if (comp == NULL) break;
if (AudioComponentGetDescription(comp, &foundDesc) != noErr) continue;
switch (foundDesc.componentType) {case kAudioUnitType_RemoteEffect:case kAudioUnitType_RemoteGenerator:case kAudioUnitType_RemoteInstrument:case kAudioUnitType_RemoteMusicEffect:// found a node...break;
}}
![Page 42: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/42.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 43: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/43.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 44: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/44.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 45: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/45.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 46: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/46.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 47: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/47.jpg)
Getting node informationNode App Discovery
RemoteAU *rau = [[RemoteAU alloc] init];rau->_desc = foundDesc;rau->_comp = comp;
AudioComponentCopyName(comp, (CFStringRef *)&rau->_name);
rau->_image = [AudioComponentGetIcon(comp, 48) retain];
rau->_lastActiveTime = AudioComponentGetLastActiveTime(comp);
[_audioUnits addObject: rau];
![Page 48: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/48.jpg)
Observing registration changesNode App Discovery
• Registrations change dynamically■ When apps installed/deleted■ When media services reset
NSNotificationCenter *nc = [NSNotificationCenter defaultCenter];[nc addObserverForName: (NSString *) kAudioComponentRegistrationsChangedNotification object: nil queue: nil usingBlock: ^(NSNotification *) { [self refreshAUList]; }];
![Page 49: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/49.jpg)
Connecting to a NodeRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 50: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/50.jpg)
Connecting to a Node
AudioUnit myAudioUnit;err = AudioComponentInstanceNew(comp, &myAudioUnit);
•Node app will be launched into background
![Page 51: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/51.jpg)
Activate audio sessionPreparing a Node
[[AVAudioSession sharedSession] setActive: YES];
• Sample rate, channel count
![Page 52: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/52.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 53: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/53.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 54: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/54.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 55: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/55.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 56: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/56.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 57: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/57.jpg)
Set stream formats
AudioStreamBasicDescription format;
format.mChannelsPerFrame = 2; // stereoformat.mSampleRate = [AVAudioSession sharedSession].sampleRate;format.mFormatID = kAudioFormatLinearPCM;format.mFormatFlags = kAudioFormatFlagsNativeFloatPacked | kAudioFormatFlagIsNonInterleaved;format.mBytesPerFrame = format.mBytesPerPacket = sizeof(Float32);format.mBitsPerChannel = 32;format.mFramesPerPacket = 1;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 0, &format, sizeof(format));
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &format, sizeof(format));
Preparing a Node
![Page 58: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/58.jpg)
Connect inputPreparing a Node
• For effects• From AudioUnit:
■ AUGraphConnectNodeInput■ kAudioUnitProperty_MakeConnection
• From callback:■ kAudioUnitProperty_SetRenderCallback
•Output
![Page 59: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/59.jpg)
DisconnectionPreparing a Node
• Can happen automatically■ Node terminates■ Host fails to render
• Instance becomes “zombie”• kAudioComponentErr_InstanceInvalidated
![Page 60: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/60.jpg)
Disconnection callbackPreparing a Node
AudioUnitAddPropertyListener(myAudioUnit, kAudioUnitProperty_IsInterAppConnected, NodeConnectionListener, self);
void NodeConnectionListener(void *userData, AudioUnit myAudioUnit, AudioUnitPropertyID, AudioUnitScope, AudioUnitElement){
UInt32 connected = 0, size = sizeof(connected);OSStatus err =AudioUnitGetProperty(myAudioUnit, kAudioUnitProperty_IsInterAppConnected, kAudioUnitScope_Global, 0, &connected, &size);if (err != noErr || connected == 0) { // Node disconnected}
}
![Page 61: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/61.jpg)
InitializationPreparing a Node
// Prepare for renderingAudioUnitInitialize(myAudioUnit);
•Host must call AudioUnitRender regularly
![Page 62: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/62.jpg)
SummaryPreparing a Node
•Activate audio session• Set stream formats• Connect audio input•Add disconnection listener• Initialize
![Page 63: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/63.jpg)
Host Rendering Node AudioRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 64: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/64.jpg)
Using AUGraphHost Rendering Node Audio
AURemoteIO
Node AppNode Audio Unit
Mixer AU
File Player AU
![Page 65: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/65.jpg)
Using AudioUnits directlyHost Rendering Node Audio
AURemoteIO
Node AppNode Audio Unit
AudioUnitRender
kAudioUnitProperty_SetRenderCallback
{ ... }
![Page 66: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/66.jpg)
•Do this after user opens the node• Icon; tap to switch
void SwitchToNode(AudioUnit node){
! NSURL *url = NULL;! UInt32 propertySize = sizeof(url);! // property is CFURLRef but NSURL is toll-free-bridged.! OSStatus err = AudioUnitGetProperty(peerAudioUnit, kAudioUnitProperty_PeerURL, kAudioUnitScope_Global, 0, &url, &propertySize);! if (err) return;!! [[UIApplication sharedApplication] openUrl: url];! [url release];
}
Switching to a Node
![Page 67: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/67.jpg)
•Do this after user opens the node• Icon; tap to switch
void SwitchToNode(AudioUnit node){
! NSURL *url = NULL;! UInt32 propertySize = sizeof(url);! // property is CFURLRef but NSURL is toll-free-bridged.! OSStatus err = AudioUnitGetProperty(peerAudioUnit, kAudioUnitProperty_PeerURL, kAudioUnitScope_Global, 0, &url, &propertySize);! if (err) return;!! [[UIApplication sharedApplication] openUrl: url];! [url release];
}
Switching to a Node
![Page 68: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/68.jpg)
•Do this after user opens the node• Icon; tap to switch
void SwitchToNode(AudioUnit node){
! NSURL *url = NULL;! UInt32 propertySize = sizeof(url);! // property is CFURLRef but NSURL is toll-free-bridged.! OSStatus err = AudioUnitGetProperty(peerAudioUnit, kAudioUnitProperty_PeerURL, kAudioUnitScope_Global, 0, &url, &propertySize);! if (err) return;!! [[UIApplication sharedApplication] openUrl: url];! [url release];
}
Switching to a Node
![Page 69: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/69.jpg)
Being Connected (for Nodes)RegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 70: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/70.jpg)
On launchBeing Connected (for Nodes)
• Can be launched into the background• Can’t start running from background•Must publish I/O unit
UIApplicationState appstate = [UIApplication sharedApplication].applicationState;_inForeground = (appstate != UIApplicationStateBackground);
![Page 71: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/71.jpg)
Handling connectionBeing Connected (for Nodes)
• kAudioUnitProperty_IsInterAppConnected•When value becomes 1:
■ Output unit has been initialized■ Set audio session active (if accessing mic)■ Start running■ Even if in the background
•AudioOutputUnitGetHostIcon
![Page 72: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/72.jpg)
Handling disconnectionBeing Connected (for Nodes)
•When kAudioUnitProperty_IsInterAppConnected becomes 0:■ Output unit has been uninitialized and stopped■ Set audio session inactive (if using mic)■ May set session active and resume running, but only if in the foreground
![Page 73: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/73.jpg)
Running State (for Nodes)
• CanStart = Connected or InForeground• Running = Connected or AppSpecificConditions
![Page 74: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/74.jpg)
StandaloneNode Rendering
AURemoteIO
Node App Engine
![Page 75: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/75.jpg)
Generator or instrumentNode Rendering
AURemoteIO
Node App Engine
Host App
![Page 76: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/76.jpg)
Generator or instrument with audio inputNode Rendering
AURemoteIO
Node App Engine
Host App
![Page 77: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/77.jpg)
EffectNode Rendering
AURemoteIO
Node App Engine
Host App
![Page 78: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/78.jpg)
Switching from Node to Host
• Same as with host: Use kAudioUnitProperty_PeerURL
![Page 79: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/79.jpg)
Stopping Audio Rendering (Host)
•AudioOutputUnitStop/AUGraphStop•AudioUnitUninitialize
■ Release resources■ Can re-initialize
•AudioComponentInstanceDispose■ When completely finished■ After node invalidated
![Page 80: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/80.jpg)
MIDIRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 81: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/81.jpg)
MIDI
• For RemoteInstrument and RemoteMusicEffect nodes■ When MIDI events tied to audio■ Sample-accurate scheduling■ Not for sync (clock/timecode)
• Complements CoreMIDI■ USB/network■ Apps that don’t support inter-app audio
![Page 82: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/82.jpg)
Immediate, unscheduledSending MIDI Events (Host)
UInt32 offsetSampleFrames = 0;const UInt8 kMIDINoteOn = 0x90;const UInt8 kMiddleC = 60;const UInt8 kVelocity = 64;MusicDeviceMIDIEvent(myAudioUnit, kMIDINoteOn, kMiddleC, kVelocity, offsetSampleFrames);
![Page 83: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/83.jpg)
ScheduledSending MIDI Events (Host)
double offsetSeconds = ...;UInt32 offsetSampleFrames = offsetSeconds * sampleRate;const UInt8 kMIDINoteOn = 0x90;MusicDeviceMIDIEvent(myAudioUnit, kMIDINoteOn, 60 /* middle C */, 64, offsetSampleFrames);AudioUnitRender(myAudioUnit, ...);
![Page 84: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/84.jpg)
Scheduled, with AUGraphSending MIDI Events (Host)
AUGraphAddRenderNotify(myGraph, MyRenderNotify, self);...OSStatus MyRenderNotify(... const AudioTimeStamp *inTimeStamp ... UInt32 inNumberFrames ...){
double offsetSeconds = ...;UInt32 offsetSampleFrames = offsetSeconds * sampleRate;const UInt8 kMIDINoteOn = 0x90;MusicDeviceMIDIEvent(myAudioUnit, kMIDINoteOn, 60 /* middle C */, 64, offsetSampleFrames);...
}
![Page 85: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/85.jpg)
MIDI CallbacksReceiving MIDI Events (Node)
typedef struct {void *userData;// see MusicDeviceMIDIEvent, MusicDeviceSysExvoid (*MIDIEventProc)(void *userData, UInt32 inStatus, UInt32 inData1, UInt32 inData2, UInt32 inOffsetSampleFrame);void (*MIDISysExProc)(void *userData, const UInt8* inData, UInt32 inLength);
} AudioOutputUnitMIDICallbacks;
![Page 86: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/86.jpg)
Receiving MIDI Events (Node)
void MyMIDIEventProc(void *userData, UInt32 inStatus, UInt32 inData1, UInt32 inData2, UInt32 inOffsetSampleFrame){
MyEngine *engine = (MyEngine *)userData;...
}
void InstallMIDICallbacks(MyEngine *engine){
AudioOutputUnitMIDICallbacks callbacks;callbacks.userData = engine;callbacks.MIDIEventProc = MyMIDIEventProc;callbacks.MIDISysExProc = NULL;AudioUnitSetProperty(myOutputUnit, kAudioOutputUnitProperty_MIDICallbacks, kAudioUnitScope_Global, 0, &callbacks, sizeof(callbacks));
}
![Page 87: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/87.jpg)
Receiving MIDI Events (Node)
void MyMIDIEventProc(void *userData, UInt32 inStatus, UInt32 inData1, UInt32 inData2, UInt32 inOffsetSampleFrame){
MyEngine *engine = (MyEngine *)userData;...
}
void InstallMIDICallbacks(MyEngine *engine){
AudioOutputUnitMIDICallbacks callbacks;callbacks.userData = engine;callbacks.MIDIEventProc = MyMIDIEventProc;callbacks.MIDISysExProc = NULL;AudioUnitSetProperty(myOutputUnit, kAudioOutputUnitProperty_MIDICallbacks, kAudioUnitScope_Global, 0, &callbacks, sizeof(callbacks));
}
![Page 88: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/88.jpg)
Receiving MIDI Events (Node)
void MyMIDIEventProc(void *userData, UInt32 inStatus, UInt32 inData1, UInt32 inData2, UInt32 inOffsetSampleFrame){
MyEngine *engine = (MyEngine *)userData;...
}
void InstallMIDICallbacks(MyEngine *engine){
AudioOutputUnitMIDICallbacks callbacks;callbacks.userData = engine;callbacks.MIDIEventProc = MyMIDIEventProc;callbacks.MIDISysExProc = NULL;AudioUnitSetProperty(myOutputUnit, kAudioOutputUnitProperty_MIDICallbacks, kAudioUnitScope_Global, 0, &callbacks, sizeof(callbacks));
}
![Page 89: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/89.jpg)
Transport and Timeline InformationRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 90: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/90.jpg)
Transport and Timeline Information
•Host is master; node can synchronize•Musical position• Transport state• Called at render time
![Page 91: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/91.jpg)
HostCallbackInfoTransport and Timeline Information
typedef struct HostCallbackInfo { void * hostUserData; HostCallback_GetBeatAndTempo beatAndTempoProc; HostCallback_GetMusicalTimeLocation musicalTimeLocationProc; HostCallback_GetTransportState transportStateProc; HostCallback_GetTransportState2 transportStateProc2;} HostCallbackInfo;
![Page 92: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/92.jpg)
Beat and tempoProviding Host Callbacks
OSStatus MyBeatAndTempo(...){
*outCurrentBeat = ...; // position in track (float)*outCurrentTempo = ...; // beats per minutereturn noErr;
}
Beat BeatTempo
![Page 93: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/93.jpg)
Musical time locationProviding Host Callbacks
OSStatus MyMusicalTimeLocation(...){
*outDeltaSampleOffsetToNextBeat = ...;*outTimeSigNumerator = ...;*outTimeSigDenominator = ...;*outCurrentMeasureDownbeat = ...;return noErr;
}
![Page 94: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/94.jpg)
Transport stateProviding Host Callbacks
OSStatus MyGetTransportState(...){
*outIsPlaying = ...;*outIsRecording = ...;*outTransportStateChanged = ...;*outCurrentSampleInTimeline = ...;*outIsCycling = ...;*outCycleStartBeat = ...;*outCycleEndBeat = ...;return noErr;
}
![Page 95: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/95.jpg)
Installing Host Callbacks
void InstallHostCallbacks(MyEngine *engine, AudioUnit myAudioUnit){
HostCallbackInfo hci;
hci.hostUserData = engine;hci.beatAndTempoProc = MyBeatAndTempo;hci.musicalTimeLocationProc = MyMusicalTimeLocation;hci.transportStateProc = NULL;hci.transportStateProc2 = MyGetTransportState;
AudioUnitSetProperty(myAudioUnit, kAudioUnitProperty_kHostCallbacks, kAudioUnitScope_Global, 0, &hci, sizeof(hci));
}
![Page 96: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/96.jpg)
Using Host Callbacks (Node)
•Get kAudioUnitProperty_HostCallbacks at connection timeHostCallbackInfo myHostCallbacks;UInt32 propertySize = sizeof(myHostCallbacks);AudioUnitGetProperty(myIOUnit, kAudioUnitProperty_kHostCallbacks, kAudioUnitScope_Global, 0, &myHostCallbacks, &propertySize);
• Call the desired host callback(s) at render time• Caution: Thread-safety
![Page 97: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/97.jpg)
Observing transport state changesUsing Host Callbacks (Node)
• Transport state listener• Called on non-render thread
AudioUnitAddPropertyListener(myIOUnit, kAudioOutputUnitProperty_HostTransportState, MyTransportStateListener, self);
![Page 98: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/98.jpg)
Remote ControlRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 99: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/99.jpg)
Audio Unit Remote Control Events
•Distinct from UIKit remote control events•Node can control host’s transport
enum {! kAudioUnitRemoteControlEvent_TogglePlayPause = 1,! kAudioUnitRemoteControlEvent_ToggleRecord = 2,! kAudioUnitRemoteControlEvent_Rewind = 3};typedef UInt32 AudioUnitRemoteControlEvent;
•Node: Use the recommended transport controls in the sample app
![Page 100: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/100.jpg)
From nodeAudio Unit Remote Control Events
•Query whether host supports them:UInt32 eventsSupported = 0, propertySize = sizeof(eventsSupported);AudioUnitGetProperty(myIOUnit, kAudioOutputUnitProperty_HostReceivesRemoteControlEvents, kAudioUnitScope_Global, 0, &eventsSupported, &propertySize);
• Send an event:AudioUnitRemoteControlEvent theControl = kAudioUnitRemoteControlEvent_ToggleRecord;AudioUnitSetProperty(myIOUnit, kAudioOutputUnitProperty_RemoteControlToHost, kAudioUnitScope_Global, 0, &theControl, sizeof(theControl));
![Page 101: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/101.jpg)
In hostAudio Unit Remote Control Events
• Set kAudioUnitProperty_RemoteControlEventListener
AudioUnitRemoteControlEventListener listenerBlock = ^(AudioUnitRemoteControlEvent event) { [self handleRemoteEvent: event]; };AudioUnitSetProperty(myAudioUnit, kAudioOutputUnitProperty_RemoteControlEventListener, kAudioUnitScope_Global, 0, &listenerBlock, sizeof(listenerBlock));
![Page 102: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/102.jpg)
DemoInter-app host, instrument and effect, with remote control
Harry Tormey
![Page 103: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/103.jpg)
Audio Session Interruptions
• The usual rules apply■ Your remote I/O has already been stopped underneath you
• In hosts, system has uninitialized your node audio units
![Page 104: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/104.jpg)
Handling Media Services Reset
•AVAudioSessionMediaServicesWereResetNotification•All inter-app audio connections are broken (component instances invalidated)•Host: Dispose node AudioUnit and AURemoteIO•Node: Dispose AURemoteIO• Proceed as if app has been launched
![Page 105: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/105.jpg)
Odds and Ends
•Multiple hosts?■ If all mixable, yes
•Multiple nodes?■ Yes
![Page 106: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/106.jpg)
Tips
•Debug node registration■ Watch console log
• Error -12985■ “Operation denied”■ Can’t start playing from the background
![Page 107: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/107.jpg)
ReviewRegisterDiscover
System ConnectConnect
Audio
MIDI
Transport/Timeline
Remote Control
Host App Node App
![Page 108: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/108.jpg)
Conclusion
• Existing apps can be converted to nodes fairly simply• Creating a host is a bit more work, but use AudioUnit APIs for power and flexibility•Use sample code from WWDC library•Make great music apps!
![Page 109: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/109.jpg)
Contact informationApple Evangelists
John GeleynseDirector, Technology [email protected]
DocumentationAV Foundation Programming Guidehttp://developer.apple.com/library/ios/#documentation/AudioVideo/Conceptual/AVFoundationPG/
Apple Developer Forumshttp://devforums.apple.com
![Page 110: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/110.jpg)
Related Sessions
Preparing and Presenting Media for Accessibility Nob HillWednesday 10:15AM
Moving to AV Kit and AV Foundation Pacific HeightsTuesday 4:30PM
Advanced Editing with AV Foundation MarinaThursday 9:00AM
What’s New in Camera Capture Nob HillWednesday 11:30AM
![Page 111: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/111.jpg)
Labs
Audio Lab Media Lab BWednesday 9:00AM
Audio Lab Media Lab BTuesday 2:00PM
![Page 112: What’s New in Core Audio for iOS · These are confidential sessions—please refrain from streaming, blogging, or taking pictures Session 602 What’s New in Core Audio for iOS](https://reader034.vdocument.in/reader034/viewer/2022042201/5ea1f1a42c83a01c12151398/html5/thumbnails/112.jpg)