pcs technical enrichment matrix dsp multimedia software architecture presented by: patrick gries 3g...
DESCRIPTION
TEM: 3 April 2002Motorola Confidential Proprietary3 Background (1 of 2) Legacy DSP code: DSP56600 or TI320C5x TI WhiteCap / Motorola WhiteCap Wally Patriot New Platform: Rainbow (StarCore SC140) No Reuse Available Design for Reuse Multimedia Features Required Talon A820 – 1 st Motorola 3G PhoneTRANSCRIPT
PCS Technical Enrichment Matrix
DSP Multimedia Software Architecture
Presented By:Patrick Gries
3G DSP Development
3 April 2002
TEM: 3 April 2002 Motorola Confidential Proprietary
2
Agenda Background 3G DSP Challenges Architectural Goals RTOS-101 Architecture Interfaces
Modem MDI IPCM data Handler API
DMP Hi-Fidelity Audio Video Memory Summary/Conclusion
TEM Goals:
RTOS
Talon Architecture
TEM: 3 April 2002 Motorola Confidential Proprietary
3
Background (1 of 2)Legacy DSP code: DSP56600 or TI320C5x
TI WhiteCap / Motorola WhiteCap Wally Patriot
New Platform: Rainbow (StarCore SC140)No Reuse AvailableDesign for ReuseMultimedia Features RequiredTalon A820 – 1st Motorola 3G Phone
TEM: 3 April 2002 Motorola Confidential Proprietary
4
Background (2 of 2)
TEM: 3 April 2002 Motorola Confidential Proprietary
5
3G DSP Challenges (1 of 6) Dual Mode Modem (GSM & WCDMA) Traditional 8 kHz Support
Echo Canceller / Noise Suppressor / Audio Shaper Common Tone Generator Speech Recognition Voice Annotation
4 Vocoders GSM Full-Rate (GSM 06.10) GSM Half-Rate (GSM 06.20) GSM Enhanced Full-Rate (GSM 06.60) Adaptive Multi-Rate (3G TS 26.090)
TEM: 3 April 2002 Motorola Confidential Proprietary
6
3G DSP Challenges (2 of 6) High-Fidelity Audio Decoders
MP3 (ISO/IEC 11172-3 & 13818-3) AAC (ISO/IEC 13818-7) MIDI WMA
Audio/Graphic Equalization Video
MPEG4 (ISO/IEC 14496-2) H.263 WMV
Playback & Streaming
TEM: 3 April 2002 Motorola Confidential Proprietary
7
3G DSP Challenges (3 of 6) Beta-level Tools - CodeWarrior Memory Footprint
Total vs. Runtime Runtime Available vs. Runtime Required
MIPS Current Drain
TEM: 3 April 2002 Motorola Confidential Proprietary
8
3G DSP Challenges (4 of 6)
2G DSP Components
3G DSP Components
TEM: 3 April 2002 Motorola Confidential Proprietary
9
3G DSP Challenges (5 of 6)
2G DSP Components
3G DSP Components
TEM: 3 April 2002 Motorola Confidential Proprietary
10
3G DSP Challenges (6 of 6)
Multimedia Component Memory
FR
HR
AMR/ EFR
CTGECS/ NSVR2
MP3 AACMIDI
MPEG4
0
20
40
60
80
100
Memory (K bytes)
TEM: 3 April 2002 Motorola Confidential Proprietary
11
Architectural Goals (1 of 2) Isolate Multimedia Subsystem from
Modem Utilize Commercial RTOS
RTXCdsp from Lineo - Minimal Configuration Process Scheduling Intertask Communication Data Movement
Hierarchical Design for Modularity Tasks - blocking Handlers – data routing Components - processing
TEM: 3 April 2002 Motorola Confidential Proprietary
12
Architectural Goals (2 of 2) Minimize Use of Global Memory Components Designed for Reuse
Single Function call No RTOS References No Hardware Interface References
Standardized Component APIs 4 pointers (input, output, control, status) Re-entrancy
TEM: 3 April 2002 Motorola Confidential Proprietary
13
RTOS-101 (1 of 6) RTOS Requirements
Pre-emptive scheduling Low overhead to meet processing
deadlines Minimal footprint
RTXCdsp from Lineo Pre-emptive scheduling Scalable services Easy-to-use kernel services Library-based
TEM: 3 April 2002 Motorola Confidential Proprietary
14
RTOS-101 (2 of 6) RTXCdsp Configuration
RTXCms only Tasks – data processing Semaphores – intertask communication Queues – data movement & intertask
comm No dedicated timers!
TEM: 3 April 2002 Motorola Confidential Proprietary
15
RTOS-101 (3 of 6)
RTXCdsp Topology task, threads, both RTXCms API
Task Context Virtual Processors
Object classes Kernel services that
operate on them ISRs
Kernel-aware Private
HW devices
Task 2…
ISR
startup
SysInit
RTXCms
API
API
Task 1
Zone 1
Zone 2
Zone 3
TEM: 3 April 2002 Motorola Confidential Proprietary
16
RTOS-101 (4 of 6)
Zone 1 Services IS_SignalSema()
Zone 3 Services KS_ExecuteTask() KS_SignalSema() KS_TestSemaW() KS_TestSemaMW() KS_PutQueueData
() KS_GetQueueData
()
TEM: 3 April 2002 Motorola Confidential Proprietary
17
RTOS-101 (5 of 6)Basic Task Pseudocode
taskA() { … task initialization, user-specified as needed for (;;) { KS_TestSemaW(MYSEMA); … perform event processing as required } }
TEM: 3 April 2002 Motorola Confidential Proprietary
18
RTOS-101 (5 of 6)
TEM: 3 April 2002 Motorola Confidential Proprietary
19
Arch - Multimedia Subsystem Perspective
TEM: 3 April 2002 Motorola Confidential Proprietary
20
Arch - Simplified Rainbow HW Platform
IPCM MCUDSP
VSAP USB
MDIASAP
PCAP
RAINBOW
speechcodec
s tereoDAC
BluetoothI/F device 123
TEM: 3 April 2002 Motorola Confidential Proprietary
21
Arch – Triggering Mechanisms (1 of 5)
datacontrol
Process X data input
data output
trigger to generate
data
“data ready” event
Basic Multimedia Element
TEM: 3 April 2002 Motorola Confidential Proprietary
22
Arch – Triggering Mechanisms (2 of 5)
Producer A data input
data output/input
trigger to generate
data
“data ready” event
Consumer B
data output
trigger to generate
data
“data ready” event
Producer / Consumer Relationship
TEM: 3 April 2002 Motorola Confidential Proprietary
23
Arch – Triggering Mechanisms (3 of 5)
ISR data input
data output/input
IRQ
“buffer full” event
Encoderdata
output
trigger to encode
dataProducer / Consumer: Feedforward Control for Encoders
TEM: 3 April 2002 Motorola Confidential Proprietary
24
Arch – Triggering Mechanisms (4 of 5)
Decoder data input
data output/input ISR
data output
IRQ
Producer / Consumer: Feedback Control for Decoders
trigger to decode
data
“buffer empty” event
TEM: 3 April 2002 Motorola Confidential Proprietary
25
Arch – Triggering Mechanisms (5 of 5) Producers/Sources
ASAP Rx Timeslots 0-3 USB-enabled mic Speech Decoder VA Playback CTG MIDI Decoder MP3 Decoder AAC Decoder
Consumers/Destinations ASAP Tx Timeslots 0-3 USB-enabled speaker VSAP TX timeslots 0 &
1 (stereo speaker, L & R)
Speech Encoder VA Record VR recognition
(frontend)
TEM: 3 April 2002 Motorola Confidential Proprietary
26
Arch - Software Reference Model
TEM: 3 April 2002 Motorola Confidential Proprietary
27
Arch - Basic MM Task Manager (input)
TEM: 3 April 2002 Motorola Confidential Proprietary
28
Arch – MDI: Immediate Processing
mdi_fn1_hdlr()
cmd_parser_task()
audio_mgr_task()
audio mgr cmd
Q
prim1_hdlr()
MDI IRQ
TEM: 3 April 2002 Motorola Confidential Proprietary
29
Arch – MDI: Deferred Processing
mdi_fn1_hdlr()
cmd_parser_task()
audio_mgr_task()
audio mgr cmd
Q
prim1_hdlr()
MDI IRQ
TEM: 3 April 2002 Motorola Confidential Proprietary
30
Arch – Task Pseudocodemanager_task() { static const SEMA semalist[] = { DATA_IN_SEMA, DATA_OUT_SEMA, CMD_QNE_SEMA, (SEMA) 0 }; SEMA cause; for(;;) { cause = KS_TestSemaMW(semalist); switch(cause) { case DATA_IN_SEMA: ... process input data break;
case DATA_OUT_SEMA: ... process output data break;
case CMD_QNE_SEMA: ... process message in command queue data break; } } }
TEM: 3 April 2002 Motorola Confidential Proprietary
31
Arch – Task Functional Grouping
Core 8 kHz audio Every 20ms CTG and VR-FE
includedHigh-fidelity audio
Every 5.8ms, 24-72ms
VideoVR - recognition
Common Attributes Command Queues SEMA-driven
processing Handler layers Isolated
components
TEM: 3 April 2002 Motorola Confidential Proprietary
32
Arch - Talon Multimedia Architecture (prelim)
TEM: 3 April 2002 Motorola Confidential Proprietary
33
Arch – Data Buffering & Routing
CTG DL 160
160DL Speech 160
VA Playback
160
Rx SSIx 1
160 Tx SSIx
task
ISR
TEM: 3 April 2002 Motorola Confidential Proprietary
34
Arch – Codec State Machine
Hybrid ISR Processes samples (private ISR) Signals semaphores (kernel-aware
ISR)Air I/F timing
GSM – every 4.615ms WCDMA – every 10ms
TEM: 3 April 2002 Motorola Confidential Proprietary
35
Arch – Codec State Machine
waiting for
encode
waiting for
decode
waiting for
decode copy
waiting for
rollover
waiting for resync
waiting for
configure
cnt == ULUL_SEMA
cnt == DLDL_SEMA
cnt == COPYcp tx buf
cnt == 160cnt = 0
cnt == 160cnt = 0
audio == ONcnt = 0
TEM: 3 April 2002 Motorola Confidential Proprietary
36
Interfaces – Modem
Data structureNot event-drivenAir interface frame reference
GSM – every 4.615 ms WCDMA – every 10 ms ISR for state variable (i.e. sample
counter) check
TEM: 3 April 2002 Motorola Confidential Proprietary
37
Interfaces – DSP-to-MCU
TEM: 3 April 2002 Motorola Confidential Proprietary
38
Interfaces – IPCM (1 of 3) Channel Control Block
Base BD pointer Next BD pointer
Buffer Descriptor mode
• DONE (D)• WRAP (W)• CONT (C)• INTERRUPT (I)• EXTENDED (EXT)• FIRST (F)• LAST (L)• ERROR (E)
count buffer pointer extended - parameter
(opt)
CCB
012
31…
mode countbuffer pointer
parameter (opt)
BD
mode countbuffer pointer
parameter (opt)
TEM: 3 April 2002 Motorola Confidential Proprietary
39
Interfaces – IPCM (2 of 3) Current Buffer
Descriptor Use 2nd only DONE = 1 CONT = 0 WRAP = 1
DSP buffer Will not wraparound 16-bit transfers
DSP buffer
mode countbuffer pointer
parameter (opt)
BD
mode countbuffer pointer
parameter (opt)
TEM: 3 April 2002 Motorola Confidential Proprietary
40
Interfaces – IPCM (3 of 3) Base Buffer Descriptor
DONE = 1 CONT = 0 WRAP = 0
Current Buffer Descriptor DONE = 1 CONT = 1 WRAP = 1
DSP buffer Will wraparound 16-bit transfers
DSP buffer
mode countbuffer pointer
parameter (opt)
BD
mode countbuffer pointer
parameter (opt)
TEM: 3 April 2002 Motorola Confidential Proprietary
41
Interfaces – Handler Complexity (1 of 2)
McCabe Number Complexity measure of software Based on control flow representation Complexity num_loops Lower is better
TEM: 3 April 2002 Motorola Confidential Proprietary
42
Interfaces – Handler Complexity (2 of 2)
Some McCabe #’s DL UL audio 7 9 speech 9 12 ctg 3 na va 6 3 ecs/ns na 1 vr2 (fe) na 8 mpeg4a 7 3--------------------------------- MP3 19 AAC 20 MIDI 30
1
2 3
McCabe Number = 3
TEM: 3 April 2002 Motorola Confidential Proprietary
43
Interfaces - Common API
Input Control Pointer pts to struct to configure how data is to be processed includes init ctrl, ptr to params, ptr to memories
Input Data Pointer linear / modulo
Output Data Pointer pts to where component should write processed data
Output Status Pointer pts to struct where processing status is to be written
NULL ptrs where appropriate
TEM: 3 April 2002 Motorola Confidential Proprietary
44
DMP – Total Memory for MM Components
total MM required
Total SRAM
unused
0
100
200
300
400
500
600
Memory (K bytes)
TEM: 3 April 2002 Motorola Confidential Proprietary
45
DMP – Memory Overbudget
Modem needs much more than 50 Kb
Does not yet include WMA/WMVMutually exclusive MM
componentsDynamic Memory Paging (DMP)
TEM: 3 April 2002 Motorola Confidential Proprietary
46
DMP - Memory for DMP Configurations
ROM
M
PEG4
AAC
MID
I
MP3
FR/V
R2FR/E
CS/H
RFR/E
CS/A
MR
RAM
RAM
0
20
40
60
80
100
120
Memory (K bytes)
TEM: 3 April 2002 Motorola Confidential Proprietary
47
DMP – dsp_data.sRAM test/init
DMP N
DMP 2
DMP 3
core
app
DMP 4
DMP 5DMP 1
……
TEM: 3 April 2002 Motorola Confidential Proprietary
48
High-Fidelity Audio
TEM: 3 April 2002 Motorola Confidential Proprietary
49
HiFi Audio – MP3 / AAC / MIDI
Input IPCM Modulo
Processing Data-driven via “transfer done” SEMA Common API structure
Output Doubly-buffered VSAP via IPCM Linear
TEM: 3 April 2002 Motorola Confidential Proprietary
50
HiFi Audio - MP3 Component
TEM: 3 April 2002 Motorola Confidential Proprietary
51
Video – MPEG4 Video
Audio-controlled decoding via timestamp
QCIF output buffer in eDRAM via IPCMLower priority than audioPreemption must handle IPCM transfers
TEM: 3 April 2002 Motorola Confidential Proprietary
52
Video – A/V Normal Processing
TEM: 3 April 2002 Motorola Confidential Proprietary
53
Video – A/V Swapping
TEM: 3 April 2002 Motorola Confidential Proprietary
54
Memory - Talon 2019 (1 of 3)
0
50000
100000
150000
200000
250000
PROG DRAM DROM DMPMemory (bytes)
2019-1007
TEM: 3 April 2002 Motorola Confidential Proprietary
55
Memory - Talon 2019 (2 of 3)
0
50000
100000
150000
200000
250000
Memory (bytes)
WCDMA ModemGSM ModemMultimediaRTOSOtherDMP
TEM: 3 April 2002 Motorola Confidential Proprietary
56
Memory - Talon 2019 (3 of 3)
0
20000
40000
60000
80000
100000
120000
Program Data RAM Data ROMMemory (bytes)
WCDMA ModemGSM ModemMultimediaRTOSOther
TEM: 3 April 2002 Motorola Confidential Proprietary
57
Summary
Flexible architecture for future features: AMR-WB WMA/WMV Video Telephony (H.263)
MM subsystem modem subsystemsMM components of RTOS & platform
TEM: 3 April 2002 Motorola Confidential Proprietary
58
Conclusions
Benefited from commercial RTOSMM componentization:
maximizes personnel resources simplifies integration achieves reuse potential