pcs technical enrichment matrix dsp multimedia software architecture presented by: patrick gries 3g...

PCS Technical Enrichment Matrix

DSP Multimedia Software Architecture

Presented By:Patrick Gries

3G DSP Development

3 April 2002

TEM: 3 April 2002 Motorola Confidential Proprietary

2

Agenda Background 3G DSP Challenges Architectural Goals RTOS-101 Architecture Interfaces

Modem MDI IPCM data Handler API

DMP Hi-Fidelity Audio Video Memory Summary/Conclusion

TEM Goals:

RTOS

Talon Architecture

TEM: 3 April 2002 Motorola Confidential Proprietary

3

Background (1 of 2)Legacy DSP code: DSP56600 or TI320C5x

TI WhiteCap / Motorola WhiteCap Wally Patriot

New Platform: Rainbow (StarCore SC140)No Reuse AvailableDesign for ReuseMultimedia Features RequiredTalon A820 – 1st Motorola 3G Phone

TEM: 3 April 2002 Motorola Confidential Proprietary

4

Background (2 of 2)

TEM: 3 April 2002 Motorola Confidential Proprietary

5

3G DSP Challenges (1 of 6) Dual Mode Modem (GSM & WCDMA) Traditional 8 kHz Support

Echo Canceller / Noise Suppressor / Audio Shaper Common Tone Generator Speech Recognition Voice Annotation

4 Vocoders GSM Full-Rate (GSM 06.10) GSM Half-Rate (GSM 06.20) GSM Enhanced Full-Rate (GSM 06.60) Adaptive Multi-Rate (3G TS 26.090)

TEM: 3 April 2002 Motorola Confidential Proprietary

6

3G DSP Challenges (2 of 6) High-Fidelity Audio Decoders

MP3 (ISO/IEC 11172-3 & 13818-3) AAC (ISO/IEC 13818-7) MIDI WMA

Audio/Graphic Equalization Video

MPEG4 (ISO/IEC 14496-2) H.263 WMV

Playback & Streaming

TEM: 3 April 2002 Motorola Confidential Proprietary

7

3G DSP Challenges (3 of 6) Beta-level Tools - CodeWarrior Memory Footprint

Total vs. Runtime Runtime Available vs. Runtime Required

MIPS Current Drain

TEM: 3 April 2002 Motorola Confidential Proprietary

8

3G DSP Challenges (4 of 6)

2G DSP Components

3G DSP Components

TEM: 3 April 2002 Motorola Confidential Proprietary

9

3G DSP Challenges (5 of 6)

2G DSP Components

3G DSP Components

TEM: 3 April 2002 Motorola Confidential Proprietary

10

3G DSP Challenges (6 of 6)

Multimedia Component Memory

FR

HR

AMR/ EFR

CTGECS/ NSVR2

MP3 AACMIDI

MPEG4

0

20

40

60

80

100

Memory (K bytes)

TEM: 3 April 2002 Motorola Confidential Proprietary

11

Architectural Goals (1 of 2) Isolate Multimedia Subsystem from

Modem Utilize Commercial RTOS

RTXCdsp from Lineo - Minimal Configuration Process Scheduling Intertask Communication Data Movement

Hierarchical Design for Modularity Tasks - blocking Handlers – data routing Components - processing

TEM: 3 April 2002 Motorola Confidential Proprietary

12

Architectural Goals (2 of 2) Minimize Use of Global Memory Components Designed for Reuse

Single Function call No RTOS References No Hardware Interface References

Standardized Component APIs 4 pointers (input, output, control, status) Re-entrancy

TEM: 3 April 2002 Motorola Confidential Proprietary

13

RTOS-101 (1 of 6) RTOS Requirements

Pre-emptive scheduling Low overhead to meet processing

deadlines Minimal footprint

RTXCdsp from Lineo Pre-emptive scheduling Scalable services Easy-to-use kernel services Library-based

TEM: 3 April 2002 Motorola Confidential Proprietary

14

RTOS-101 (2 of 6) RTXCdsp Configuration

RTXCms only Tasks – data processing Semaphores – intertask communication Queues – data movement & intertask

comm No dedicated timers!

TEM: 3 April 2002 Motorola Confidential Proprietary

15

RTOS-101 (3 of 6)

RTXCdsp Topology task, threads, both RTXCms API

Task Context Virtual Processors

Object classes Kernel services that

operate on them ISRs

Kernel-aware Private

HW devices

Task 2…

ISR

startup

SysInit

RTXCms

API

API

Task 1

Zone 1

Zone 2

Zone 3

TEM: 3 April 2002 Motorola Confidential Proprietary

16

RTOS-101 (4 of 6)

Zone 1 Services IS_SignalSema()

Zone 3 Services KS_ExecuteTask() KS_SignalSema() KS_TestSemaW() KS_TestSemaMW() KS_PutQueueData

() KS_GetQueueData

()

TEM: 3 April 2002 Motorola Confidential Proprietary

17

RTOS-101 (5 of 6)Basic Task Pseudocode

taskA() { … task initialization, user-specified as needed for (;;) { KS_TestSemaW(MYSEMA); … perform event processing as required } }

TEM: 3 April 2002 Motorola Confidential Proprietary

18

RTOS-101 (5 of 6)

TEM: 3 April 2002 Motorola Confidential Proprietary

19

Arch - Multimedia Subsystem Perspective

TEM: 3 April 2002 Motorola Confidential Proprietary

20

Arch - Simplified Rainbow HW Platform

IPCM MCUDSP

VSAP USB

MDIASAP

PCAP

RAINBOW

speechcodec

s tereoDAC

BluetoothI/F device 123

TEM: 3 April 2002 Motorola Confidential Proprietary

21

Arch – Triggering Mechanisms (1 of 5)

datacontrol

Process X data input

data output

trigger to generate

data

“data ready” event

Basic Multimedia Element

TEM: 3 April 2002 Motorola Confidential Proprietary

22

Arch – Triggering Mechanisms (2 of 5)

Producer A data input

data output/input

trigger to generate

data

“data ready” event

Consumer B

data output

trigger to generate

data

“data ready” event

Producer / Consumer Relationship

TEM: 3 April 2002 Motorola Confidential Proprietary

23

Arch – Triggering Mechanisms (3 of 5)

ISR data input

data output/input

IRQ

“buffer full” event

Encoderdata

output

trigger to encode

dataProducer / Consumer: Feedforward Control for Encoders

TEM: 3 April 2002 Motorola Confidential Proprietary

24

Arch – Triggering Mechanisms (4 of 5)

Decoder data input

data output/input ISR

data output

IRQ

Producer / Consumer: Feedback Control for Decoders

trigger to decode

data

“buffer empty” event

TEM: 3 April 2002 Motorola Confidential Proprietary

25

Arch – Triggering Mechanisms (5 of 5) Producers/Sources

ASAP Rx Timeslots 0-3 USB-enabled mic Speech Decoder VA Playback CTG MIDI Decoder MP3 Decoder AAC Decoder

Consumers/Destinations ASAP Tx Timeslots 0-3 USB-enabled speaker VSAP TX timeslots 0 &

1 (stereo speaker, L & R)

Speech Encoder VA Record VR recognition

(frontend)

TEM: 3 April 2002 Motorola Confidential Proprietary

26

Arch - Software Reference Model

TEM: 3 April 2002 Motorola Confidential Proprietary

27

Arch - Basic MM Task Manager (input)

TEM: 3 April 2002 Motorola Confidential Proprietary

28

Arch – MDI: Immediate Processing

mdi_fn1_hdlr()

cmd_parser_task()

audio_mgr_task()

audio mgr cmd

Q

prim1_hdlr()

MDI IRQ

TEM: 3 April 2002 Motorola Confidential Proprietary

29

Arch – MDI: Deferred Processing

mdi_fn1_hdlr()

cmd_parser_task()

audio_mgr_task()

audio mgr cmd

Q

prim1_hdlr()

MDI IRQ

TEM: 3 April 2002 Motorola Confidential Proprietary

30

Arch – Task Pseudocodemanager_task() { static const SEMA semalist[] = { DATA_IN_SEMA, DATA_OUT_SEMA, CMD_QNE_SEMA, (SEMA) 0 }; SEMA cause; for(;;) { cause = KS_TestSemaMW(semalist); switch(cause) { case DATA_IN_SEMA: ... process input data break;

case DATA_OUT_SEMA: ... process output data break;

case CMD_QNE_SEMA: ... process message in command queue data break; } } }

TEM: 3 April 2002 Motorola Confidential Proprietary

31

Arch – Task Functional Grouping

Core 8 kHz audio Every 20ms CTG and VR-FE

includedHigh-fidelity audio

Every 5.8ms, 24-72ms

VideoVR - recognition

Common Attributes Command Queues SEMA-driven

processing Handler layers Isolated

components

TEM: 3 April 2002 Motorola Confidential Proprietary

32

Arch - Talon Multimedia Architecture (prelim)

TEM: 3 April 2002 Motorola Confidential Proprietary

33

Arch – Data Buffering & Routing

CTG DL 160

160DL Speech 160

VA Playback

160

Rx SSIx 1

160 Tx SSIx

task

ISR

TEM: 3 April 2002 Motorola Confidential Proprietary

34

Arch – Codec State Machine

Hybrid ISR Processes samples (private ISR) Signals semaphores (kernel-aware

ISR)Air I/F timing

GSM – every 4.615ms WCDMA – every 10ms

TEM: 3 April 2002 Motorola Confidential Proprietary

35

Arch – Codec State Machine

waiting for

encode

waiting for

decode

waiting for

decode copy

waiting for

rollover

waiting for resync

waiting for

configure

cnt == ULUL_SEMA

cnt == DLDL_SEMA

cnt == COPYcp tx buf

cnt == 160cnt = 0

cnt == 160cnt = 0

audio == ONcnt = 0

TEM: 3 April 2002 Motorola Confidential Proprietary

36

Interfaces – Modem

Data structureNot event-drivenAir interface frame reference

GSM – every 4.615 ms WCDMA – every 10 ms ISR for state variable (i.e. sample

counter) check

TEM: 3 April 2002 Motorola Confidential Proprietary

37

Interfaces – DSP-to-MCU

TEM: 3 April 2002 Motorola Confidential Proprietary

38

Interfaces – IPCM (1 of 3) Channel Control Block

Base BD pointer Next BD pointer

Buffer Descriptor mode

• DONE (D)• WRAP (W)• CONT (C)• INTERRUPT (I)• EXTENDED (EXT)• FIRST (F)• LAST (L)• ERROR (E)

count buffer pointer extended - parameter

(opt)

CCB

012

31…

mode countbuffer pointer

parameter (opt)

BD

mode countbuffer pointer

parameter (opt)

TEM: 3 April 2002 Motorola Confidential Proprietary

39

Interfaces – IPCM (2 of 3) Current Buffer

Descriptor Use 2nd only DONE = 1 CONT = 0 WRAP = 1

DSP buffer Will not wraparound 16-bit transfers

DSP buffer

mode countbuffer pointer

parameter (opt)

BD

mode countbuffer pointer

parameter (opt)

TEM: 3 April 2002 Motorola Confidential Proprietary

40

Interfaces – IPCM (3 of 3) Base Buffer Descriptor

DONE = 1 CONT = 0 WRAP = 0

Current Buffer Descriptor DONE = 1 CONT = 1 WRAP = 1

DSP buffer Will wraparound 16-bit transfers

DSP buffer

mode countbuffer pointer

parameter (opt)

BD

mode countbuffer pointer

parameter (opt)

TEM: 3 April 2002 Motorola Confidential Proprietary

41

Interfaces – Handler Complexity (1 of 2)

McCabe Number Complexity measure of software Based on control flow representation Complexity num_loops Lower is better

TEM: 3 April 2002 Motorola Confidential Proprietary

42

Interfaces – Handler Complexity (2 of 2)

Some McCabe #’s DL UL audio 7 9 speech 9 12 ctg 3 na va 6 3 ecs/ns na 1 vr2 (fe) na 8 mpeg4a 7 3--------------------------------- MP3 19 AAC 20 MIDI 30

1

2 3

McCabe Number = 3

TEM: 3 April 2002 Motorola Confidential Proprietary

43

Interfaces - Common API

Input Control Pointer pts to struct to configure how data is to be processed includes init ctrl, ptr to params, ptr to memories

Input Data Pointer linear / modulo

Output Data Pointer pts to where component should write processed data

Output Status Pointer pts to struct where processing status is to be written

NULL ptrs where appropriate

TEM: 3 April 2002 Motorola Confidential Proprietary

44

DMP – Total Memory for MM Components

total MM required

Total SRAM

unused

0

100

200

300

400

500

600

Memory (K bytes)

TEM: 3 April 2002 Motorola Confidential Proprietary

45

DMP – Memory Overbudget

Modem needs much more than 50 Kb

Does not yet include WMA/WMVMutually exclusive MM

componentsDynamic Memory Paging (DMP)

TEM: 3 April 2002 Motorola Confidential Proprietary

46

DMP - Memory for DMP Configurations

ROM

M

PEG4

AAC

MID

I

MP3

FR/V

R2FR/E

CS/H

RFR/E

CS/A

MR

RAM

RAM

0

20

40

60

80

100

120

Memory (K bytes)

TEM: 3 April 2002 Motorola Confidential Proprietary

47

DMP – dsp_data.sRAM test/init

DMP N

DMP 2

DMP 3

core

app

DMP 4

DMP 5DMP 1

……

TEM: 3 April 2002 Motorola Confidential Proprietary

48

High-Fidelity Audio

TEM: 3 April 2002 Motorola Confidential Proprietary

49

HiFi Audio – MP3 / AAC / MIDI

Input IPCM Modulo

Processing Data-driven via “transfer done” SEMA Common API structure

Output Doubly-buffered VSAP via IPCM Linear

TEM: 3 April 2002 Motorola Confidential Proprietary

50

HiFi Audio - MP3 Component

TEM: 3 April 2002 Motorola Confidential Proprietary

51

Video – MPEG4 Video

Audio-controlled decoding via timestamp

QCIF output buffer in eDRAM via IPCMLower priority than audioPreemption must handle IPCM transfers

TEM: 3 April 2002 Motorola Confidential Proprietary

52

Video – A/V Normal Processing

TEM: 3 April 2002 Motorola Confidential Proprietary

53

Video – A/V Swapping

TEM: 3 April 2002 Motorola Confidential Proprietary

54

Memory - Talon 2019 (1 of 3)

0

50000

100000

150000

200000

250000

PROG DRAM DROM DMPMemory (bytes)

2019-1007

TEM: 3 April 2002 Motorola Confidential Proprietary

55

Memory - Talon 2019 (2 of 3)

0

50000

100000

150000

200000

250000

Memory (bytes)

WCDMA ModemGSM ModemMultimediaRTOSOtherDMP

TEM: 3 April 2002 Motorola Confidential Proprietary

56

Memory - Talon 2019 (3 of 3)

0

20000

40000

60000

80000

100000

120000

Program Data RAM Data ROMMemory (bytes)

WCDMA ModemGSM ModemMultimediaRTOSOther

TEM: 3 April 2002 Motorola Confidential Proprietary

57

Summary

Flexible architecture for future features: AMR-WB WMA/WMV Video Telephony (H.263)

MM subsystem modem subsystemsMM components of RTOS & platform

TEM: 3 April 2002 Motorola Confidential Proprietary

58

Conclusions

Benefited from commercial RTOSMM componentization:

maximizes personnel resources simplifies integration achieves reuse potential