embed your ideas

Embedded Software Made Easy

Embed your Ideas

A connected chief hat reading you a recipe you selected by voice control and waiting you at each step with a voice signal before continuing to the next one.

Let’s take an example

The following content is • Under the software point-of-view. • Is simplified for a broader audience. • Opinionated by 25 years of farjump’s team

experience.

Warning

Embedded Software Made Easy 4

Before we get started: Hardware vs Software

Hardware

General-PurposeSingle-Purpose

Hardware

CPUMCUASIC GPU

General-Purpose

Single-Purpose

I/O Peripheral

Hardware

CPUMCUASIC GPU

General-Purpose

Single-Purpose

I/O Peripheral

May contain a General-Purpose CPU

Hardware

ComputingI/O

Hardware

ASIC GPU

Computing

I/O Peripheral

Hardware

CPUMCUASIC GPU

Computing

I/O Peripheral

Hardware

CPUMCUASIC GPU

Computing

The Hardware/Software Interface (HSI)

I/O Peripheral

Hardware

CPUMCUASIC GPU

Computing

We (the software people) are here

I/O Peripheral

Hardware

CPUMCUASIC GPU

Computing

I/O Peripheral

Managed by internal (embedded) software

Hardware

CPUMCUASIC GPU

Computing

I/O Peripheral

Managed by internal (embedded) softwareExternally controlled by

Computing Hardware

System-on-Chip (SoC)

T h e g e n e r a l - p u r p o s e computing hardware you deal with is in fact compound of many specialised sub-units.

I/OCoreI/O

Tree map of usual sub-units in a SoC

I/OCoreI/O

Tree map of usual sub-units in a MCU

Memory

System-on-Chip (SoC)

T h e g e n e r a l - p u r p o s e computing hardware you deal with is in fact compound of many specialised sub-units.

I/OCoreI/O

Tree map of usual sub-units in a SoC

Somehow, this is your software which specialises it. Once it is

embedded in a closed system, its purpose is fixed.

I/OCoreI/O

Tree map of usual sub-units in a MCU

Memory

Example of an ARM SoC

Example of the Intel 8021 Microcontroller

Level of Software

Abstraction

Hardware

Low-Level Software

Level of Software

Abstraction

Hardware

Low-Level Software

FirmwareBare-metal

Level of Software

AbstractionSystem Call

Operating System

Hardware

Low-Level Software

FirmwareBare-metal

Level of Software

Abstraction

System Call

Runtime SystemLibrary

Operating System

Hardware

OS Service

Low-Level Software

FirmwareBare-metal

Level of Software

Abstraction

System Call

Program

Operating System

Hardware

OS Service

Low-Level Software

FirmwareBare-metal

Level of Software

Abstraction

System Call

Program

Operating System

Hardware

Be very careful with that…

OS Service

Challenge #1 - How do I know what should be software and hardware?

We need to break down the idea.

Recipe Search Recipe ReadingPower Button

Note that these are the user interfaces…

Is that enough…?

Recipe Search

Speech Recognition

Recipe Search

Text-to-Speech

Speech Recognition

Recipe Search

Text-to-Speech

Microphone

Speech Recognition

Speaker

Recipe Search

Text-to-Speech

Microphone

Speech Recognition

Speaker

Recipe Search

Text-to-Speech

Microphone

Hardware

Software

Hardware

Software

Recipe Reading

Speech Recognition

Speaker

Step-by-Step Processing

Text-to-Speech

Microphone

Software

Hardware

Power Button

Boot Usually handled by an OS driver

Power On Power Off

Power Unit

Power Events

Usually handled by a firmware and

bootloader

The basic general picture for now

Drivers

Power Button Speaker Microphone

Text-to-Speech Speech Recognition

Recipe Search Recipe Reading

Hardware

Drivers

Hardware

Drivers

Hardware

Computing

Drivers

Software

Hardware

Computing

Drivers

Software

Hardware

Computing

Drivers

Executed by a CPU

Software

Hardware

Computing

Drivers

Managed by drivers

Executed by a CPU

Software

Hardware

Computing

Drivers

But we still don’t know what it exactly looks like…

Software

Hardware

Computing

Drivers

Which exact I/Os?

Software

Hardware

Computing

Drivers

Which exact CPU? Which exact I/Os?

We need to scale the hardware to our precise needs.

Challenge #2 - How do I select the hardware?

We need to scale the hardware to our precise needs.

Having a « chicken and egg » problem?

Use an empirical approach by prototyping starting from the first observations & deductions. Then test & evaluate and repeat further specifying your hardware until you reach your local optimum.

When you have no idea of a starting point, start with an XXL superset of hardware, rich enough to run your idea, and create a functional prototype.

Note that:• This process can be endless. It only depends on

what you are trying to accomplish. So be as lean as possible here ;-)

Note that:• This process can be endless. It only depends on

what you are trying to accomplish. So be as lean as possible here ;-)

• This is the moment where you may start doing hardware development in parallel of your software development: once the hardware chosen, the hardware team keeps further specifying it while you can start the software development.

In any case, the overall goal here is to make compromises between:

• Your physical constraints (space, weight, power, etc.).

• Your physical constraints (space, weight, power, etc.).• Your I/O requirements, including the interfaces with the

CPU.• Your computing requirements.

And only targeting the current stage of your product:

1. Research

2. Development

1. Research

2. Development

3. Production

The IoT gives us a very unique chance in the embedded world: it has the same I/O interfaces and peripheral as our mainstream general-purpose personal computers.

Which means that you can connect your IoT I/O peripherals to your own workstation to start prototyping!

Functional Prototype

Your OS

Your Workstation

Hardware

Software

Which means that you can connect your IoT I/O peripherals to your own workstation to start prototyping!

Functional Prototype

Your OS

Your Workstation

Hardware

Software

Assuming your OS has the driver (otherwise

implement it?)…

Going back to our chief hat!

I/O Requirements

It is made of single-purpose hardware which are selected according to their characteristics and our requirements.

I/OsPower Button Speaker Microphone

It is made of single-purpose hardware which are selected according to their characteristics and our requirements.

A push button A speaker for human ears…

A mic for human sounds…

It gives us our first CPU requirements:

It gives us our first CPU requirements:• They must be somehow connected to the CPU

using their interfaces (SPI, USB, PCIe…).

It gives us our first CPU requirements:• They must be somehow connected to the CPU

using their interfaces (SPI, USB, PCIe…).• They have bandwidth requirements of several

kbits/s.

Computing Requirements

Select a processor according to: • The kind of software you need to execute. • Your I/Os requirements. • Your physical constraints (power

consumption, temperature, etc.).

There are two main kind of processors:

There are two main kind of processors:• Computing-oriented: fast CPU with lots of

fast memory for CPU-bound programs (e.g., image processing).

There are two main kind of processors:• Computing-oriented: fast CPU with lots of

fast memory for CPU-bound programs (e.g., image processing).

• I/O-oriented: large I/O bandwidth, optimised I/O bridge for I/O-bound programs (e.g., a web server).

• CPU-bound programs scale according to the computing power of the CPU.

• I/O-bound programs scale according to the power of the CPU required to manage these I/Os.

So what kind of software do we run?

Drivers

Recipe Search Recipe ReadingAt run-time, most of the execution time will be probably spent in the speech algorithms. Which makes this software CPU-bound.

Drivers

Recipe Search Recipe ReadingOr not? Speech recognition is complex enough to require a huge amount of computing power we can’t afford in our embedded system. So let’s make benefit from the « I » of IoT and let’s implement this unit in the cloud.

Drivers

Recipe Search Recipe ReadingSo now, most of its execution time is spent waiting for I/Os to complete: sending requests to a speech server in the cloud and driving the speakers & microphone. Our software is I/O-bound.

And don’t forget about the new hardware and

software now required:

Software

Hardware

Computing

Drivers

Network Interface Controller

Network Stack

Our CPU should include:

Our CPU should include:• The interfaces required to connect with our I/

O peripherals, unless already present in the selected CPU/SoC/MCU.

Our CPU should include:• The interfaces required to connect with our I/

O peripherals, unless already present in the selected CPU/SoC/MCU.

• Enough computing and I/O power to run our peripherals and our software fast enough for audio processing.

Running benchmarks are necessary when you have to further refine your assumptions to get closer to the perfect hardware set.

Running benchmarks in a bare-metal execution environment is a good practice to:

• Be alone in the CPU and get rid of hardware side-effects coming from concurrent or parallel software.

• Be at the lowest software abstraction level, above the HSI, to fully understand what’s going on during the benchmark.

• Do whatever you want.

To do so…:

To do so…:• Select a JTAG probe compatible with your

CPU• affordable

CPU• affordable• implementable in less than a week

CPU• affordable• implementable in less than a week• with documentation & support

Or you can use instead Alpha, a GDB server, which brings:

• the ability to run standard C (malloc, fprintf, etc.) programs bare metal with the support of GDB.

• a bare metal execution environment ready-to-use.

Without any single JTAG probe.

Challenge #3 - How do I choose the right software architecture?

Low-Level Software

FirmwareBare-metal

Level of Software

Abstraction

System Call

Program

Operating System

OS Service

By understanding the main roles in this diagram:

• Started by the hardware from a read-only memory (flash, etc.).

• Given the constraints of its hardware environment, it is usually reserved for very special cases like booting hardware units (CPU, GPU, etc.).

Firmware

Bare-Metal

• Started by a firmware.• No abstraction but the one performed by the firmware(s).• Lowest-level software, right next to the HSI.• Doesn’t use any OS but usually libraries provided in the CPU/

SoC/MCU SDK, from simple helper functions to full network stacks.

Bare-Metal

• Started by a firmware.• No abstraction but the one performed by the firmware(s).• Lowest-level software, right next to the HSI.• Doesn’t use any OS but usually libraries provided in the CPU/

SoC/MCU SDK, from simple helper functions to full network stacks.

Used when:

✓ There is not enough memory for a complete OS.

✓ High-performance computing software.

Bare-Metal

Operating System

• Started by an OS-loader.

The two sides of an operating system:

1. Manages the hardware.

2. Provides a software abstraction of it with its own standard or specific interface.

Operating System

Like any other software, consider its features and use the one that best matches your needs.

Operating System

Real-Time Support

Safety Security Standard API

Standard API

Driver APIs

Tooling

• Started by the OS. • Benefits from the abstraction of its language

library, runtime, shared libraries, etc.

Program

Use any language you like, the one you feel confident with to implement your features…

Program

GoPython

NodeJS

Program

GoPython

NodeJS

JavaCorrectly managing errors is a feature…

…that some languages provide built-in!

Chief Hat Program

Our connected chief hat requires enough high-level features, such as a network stack or a power event handler, that are good candidates to start at least prototyping using Linux on a simple ARM SoC include a network controller (e.g., wifi). The main advantage of using Linux being its amazing development tooling…

Network Interface Controller

Network StackARM Kernel Drivers

Challenge #4 - How do I do embedded programming?

Hardware

HardwareYour Workstation

Any kind of link

Called the « target »

Any kind of link

Called the « target »Called the

« remote » link

Any kind of link

Called the « target »Called the « host »Called the

« remote » link

Any kind of link

« remote » link

Cross-Toolchain

Any kind of link

« remote » link

Cross-Toolchain

Tools to be executed on your workstation to manipulate « target » software.

Any kind of link

« remote » link

Cross-Compiler

Any kind of link

« remote » link

Cross-Compiled Program

Any kind of link

Not working? Do it again...

Challenge #5 - How do I make embedded development simple?

It’s still such a « hard » world…

In such « hard » world…

Can we make it easier for Bob, its

teammates, its boss?

Firstly…

Cross-GDB

GDB is the recommended tool. It allows to embed, debug and test the program, in one single amazing tool.

Firstly…

Cross-GDB

Debug Informations of

the Cross-Compiled Program

Firstly…

Cross-GDB

Firstly…

Cross-GDB

Firstly…

GDB Debug Protocol

Cross-GDB

Firstly…

The program is dynamically debugged, so no more static logs/

prints everywhere in the code

GDB Debug Protocol

Cross-GDB

Firstly…

The program is dynamically debugged, so no more static logs/

prints everywhere in the code

But also the entire system, including its

OS, drivers, and other processes.

GDB Debug Protocol

And finally, how would you find mistakes before they become bugs? Agile development? Test-Driven Development? Continuous Integration?

Dedicate your time and energy to your product using our second tool, Lab, which automates the setup of a target and its interfaces. Useful even on a day-to-day basis for your tedious manual setups of the targets and to be able later on to run automatic tests from a continuous integration server.

Challenge #6 - How do I go in production?

Hardware

Program in Read-Only Memory

HardwareProduction Test Bench

Any kind of link

In the chief hat case, the network controller could be used.

Any kind of linkAlpha

Any kind of link

Cross-GDBAlpha

Any kind of linkDebug

Informations of Program in Read-

Only Memory

Cross-GDBAlpha

Hardware test benches are expensive and are a whole project of their own.

We recommend also using Lab to make it as much software as possible:

• Connect your hardware to your test instruments.

• Connect your instruments to Lab and you are down with hardware, you can continue fully software…

Simplicity is prerequisite of reliability

Edsger W. Dijkstra Computer Science Pioneer

linkedin.com/in/guerrajulio

Have Fun & Good Luck!

github.com/farjump/raspberry-pi

hello@farjump.io

embed your ideas

Software

how to embed a picture into your digital portfolio on wiki...

how to embed a glog in your blog

how to embed a glog in your blog 2015

five big ideas to develop mastery - mathematical...

your ideas, your initiatives

10 ways to embed well-being into remote learning...

embed exif data into your images for free

how to upload presentations in slideshare and embed in your...

embed solvvy into your mobile app mobile... ·...

how to embed dailymotion videos on tumblr with your...

video embed codes - presskit.we-vibe.com · tango by...

upload videos to vimeo and embed the vimeo videos in your...

create a voki avatar talking...

250ok analytics one pager - amazon s3send your campaigns use...

how to shoot edit and embed video on your website or ebook

please enjoy this article - amazon s3 · please enjoy this...

embed data and information quality into your organisation's...

how to embed an instagram photo into your wordpress blog

create your qr codes creative designs embed logo multimedia

embed gooddata analytics in your force.com app