bluecoin: the robotic ear - home - stmicroelectronics · 2021. 7. 19. · •bluetooth 4.1,...
TRANSCRIPT
![Page 1: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/1.jpg)
BlueCoin: The Robotic Ear
Edoardo GallizioMicrosystems Product Marketing Manager
STMicroelectronics
![Page 2: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/2.jpg)
BlueCoin: The Robotic Ear
SoundLocalization
Embedded Processing
Orientation, 3D TrackingAbsolute Altitude
Acoustic Beamforming & Source Localization
Bluetooth Low Energy Connectivity
![Page 3: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/3.jpg)
BlueCoin: The Robotic EarAugmented hearing, ToF and motion sensing• Features
– Advanced audio processing• Sound Source Localization• Beamforming
– Wide band audio over BLE (BlueVoice)– 10 DOF Sensor fusion + Contextual Awareness
• Inertial, environmental, MEMS Microphones
– Complete Reference Design• SW Compatible with STM32 ODE• CoinStation + small Cradle
25mm
Sense
Connect
![Page 4: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/4.jpg)
BlueCoin: The Robotic Ear• Main components
– STM32F446• ARM Cortex-M4F@180MHz - 128KB RAM
– u4 Microphone Array (4x MP34DT04)– Bluetooth-Low-Energy radio (BlueNRG-MS)
• Bluetooth 4.1, multiple role simultaneously
– 6+3 axis inertial module (LSM6DSM+LSM303AGR)– Absolute pressure sensor (LPS22HB)– 2x VL53L0X Time-of-Flight (ToF) ranging sensor
BlueCoin + battery
![Page 5: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/5.jpg)
BlueCoin: Open Software Support
4 x Microphones
Accelerometer + Gyroscope
BLE
STM32F4 ARM Cortex-M4
HW-SNS-ALLMEMS1: STM32 ODE function pack for IoT node with BLE connectivity, digital microphone, environmental and motion sensors
Sense
E-compass
Pressure SensorConnect
Processing
![Page 6: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/6.jpg)
BlueCoin: STEVAL-BCNKT01V1
Sensors
Ultra Low PowerConnectivity
Low-Power MCU
4 x MP34DT04
LPS22HB
LSM6DSMLSM303AGR
STM32F446
BlueNRG-MS
Motion MEMS
Environmental sensors
MEMS microphone array
High-Performance brain
Sensor fusion
Bluetooth Smart
Power/Battery Management Li-ion Battery Charger STBC03J
![Page 7: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/7.jpg)
BlueCoin: The Robotic Ear
4 x MP34DT04-C1
BOTTOM VIEW TOP VIEW
LSM6DSM/L
ExpansionConnectors
2 x Buttons
STBC03J
SMD Antenna
BLUENRG-MS
STM32F446
LPS22HB
LSM303AGR 8 x LEDS
Ø 2
5 m
m
Expansions:I2S, I2C, USB, TIMx,
CAN, UART, SWD, SDIO
BALF-NRG-01D3
![Page 8: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/8.jpg)
Enabling Multiple SW Libraries• Example project in source code built on STM32Cube software technology• Includes acoustic Beam Forming, Echo Cancellation, and Source Localization. • Immediate test and performance evaluation
Beamforming
Acoustic Echo Cancellation
reference audio
Source Localization4-MEMSmicrophones array
![Page 9: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/9.jpg)
Audio SW IP and Eco-system
AudioAnalytics
StatisticalDereverberation
Auto Gain Control
Trigger ASR
Noise Reduction
reference
- Voice Activity Detection - Statistical moments- Noise estimation
3rd party ASR
cloudembedded
MEMS microphone array
osxAcousticSL
osxAcousticBF osxAcousticEC
Each osxAcoustic library may be easily replaced by 3rd party SW IP
All are released under free evaluation and production licensing
![Page 10: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/10.jpg)
Sound Source Localization LibraryTwo algorithms implemented
• XCORR• GCC-PHAT
A simple Voice Activity Detector is included, based on energy threshold.
– Avoids false recognitions in case of low signal energy
Range 2 microphones cover a range of 180°
4 microphones cover a range of 360°
![Page 11: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/11.jpg)
Acoustic Echo CancellationRemoves echo of playback audio in speech capture application
AEC(estimates
room reverberation)
Reverberant Room
Known Audio Sourcee. g. music / voice
Single Microphone application
STM32 is connected to both the microphone and the loudspeaker
The Open.AUDIO AEC library is an optimized STM32 port based on the Open Source project Speex: http://www.speex.org/
![Page 12: BlueCoin: The Robotic Ear - Home - STMicroelectronics · 2021. 7. 19. · •Bluetooth 4.1, multiple role simultaneously – 6+3 axis inertial module (LSM6DSM+LSM303AGR) – Absolute](https://reader035.vdocument.in/reader035/viewer/2022071518/613c30914c23507cb63538d4/html5/thumbnails/12.jpg)
BlueCoin – Audio Augmented for IoT
BlueCoin Development
Platform
Reference Software and Applications
Cloud-based Conversational
Interface
Advanced Audio Capture frontend