can2wifi: project report in car multimedia project … · usually limited to audio streaming and...

Technical Report 2013-01 ICM Labs | Faculty of Computer Science | University of Applied Sciences Darmstadt

1

Abstract—Mobile devices like tablets and smartphones are al-

ready part of our everyday life. This applies also to cars with their infotainment systems. The communication between both is usually limited to audio streaming and support of hands-free equipment. Access to the extensive sensor information of the vehicle is not available. This founded the motivation for the 2013 student development project at the In-Car-Multimedia Labs of the Faculty of Computer Science at the University of Applied Science Darmstadt.

Within a block course a group of BSc students developed a sys-tem that interconnects Android based mobile devices with the fieldbus system of a BMW 5 series. Centerpiece of the imple-mented prototype is an Intel Atom based embedded system that acts as gateway to the automotive fieldbus and managing inter-preter.

This report details the problem definition, the organizational context for the students, the developed system architecture and the implemented results.

I. INTRODUCTION HE ICM LABS offer BSc students of the Faculty of Com-puter Science at h_da University of Applied Science the

opportunity to participate in a system development project within the non-lecture period in late summer. 2013 was the fourth time where a group of 20 advanced bachelor’s level students attended this elective course related to embedded automotive systems. This year’s topic is “CAN2WiFi” (Controller Area Network to Wireless Fidelity). The challenge is to realize the provisioning of automotive sensor data and information available on the CAN-bus via a wireless local area network (IEEE 802.11b/g) and process and visualize the vehicle data using a mobile telephone or Internet tablet based on the Google Android operating system. The scope for the project is to build a foun-dation for advanced Android APPs, incorporating sensor data ICM Technical Report ICM-TR 2013-01; First Revision; 28. November 2013.

This report provides information on the 2013 BSc system development project at the ICM labs, Faculty of Computer Science, University of Applied Sciences Darmstadt.

© 2013. All rights reserved. The authors retain all copyrights in any text, graphic images and photos in this report. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechan-ical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the authors.

A. Knirsch, FB Informatik, Hochschule Darmstadt, Haardtring100, 64295 Darmstadt, Germany (e-mail: [email protected]).

Prof. Dr.-Ing. J. Wietzke, FB Informatik, Hochschule Darmstadt, Haar-dtring 100, 64295 Darmstadt, Germany (e-mail: [email protected]).

for more accurate positioning, advanced driver information, and personalized assistance functionalities based on real-time vehicle information. The desired outcome is a functional pro-totype usable within a real-life scenario. Therefore the follow-ing minimal requirements for the targeted system are defined:

(1) A mobile device running an Android operating sys-tem shall display a digital instrument cluster without any notable latency compared to the vehicles built-in instrument cluster.

(2) The digital instrument cluster shall visualize at least the current speed (velocity), the engine’s RPM (rounds per minute) and direction indicators.

(3) The digital instrument cluster shall visualize a meas-ure for the current horsepower, derived from the ve-hicles acceleration (see [We13]).

(4) The mobile device shall be connected to the vehicle (or a system attached to the vehicle) using WiFi, in particular IEEE 802.11.

(5) An embedded system shall act as gateway to the ve-hicular fieldbus (CAN).

(6) The embedded system shall provide connectivity to mobile devices by acting as WiFi access point.

(7) The embedded system shall handle multiple simulta-neous connected mobile devices.

The system components are provided, consisting of an Intel Atom dual-core based DE2i-150 FPGA Development Kit (see Fig. 2) with preinstalled GNU/Linux operating system and

CAN2WiFi: Project Report In-Car Multimedia Project System Development

Andreas Knirsch, Prof. Dr.-Ing. Joachim Wietzke ICM labs, Faculty of Computer Science, University of Applied Sciences Darmstadt

T

Fig. 1. Android tablet and mobile within an automotive context.


2

built-in WiFi facilities, a PEAK CAN controller to connect to the vehicular fieldbus, and both a simulating environment to playback CAN traces and a BMW 5 series for road trials.

Some experience with the topic is already available, as result of preliminary investigations and preceding projects in form of functional software and project documentation. In addition to the functional part of the project, the students are given the chance to gather experience related to the man-agement and intercommunication within a software project. Therefore the course is split into teams to develop distinct but interdependent system modules, which have to be integrated to achieve the required functionality. This means, all develop-ment teams have to deliver functional, compatible and in-teroperable system modules within a fixed schedule. That implies the need to solve problems within teams as well as beyond the module boundaries.

II. ORGANIZATIONAL ENVIRONMENT The participating students receive 7.5 ETCS and grades re-spectively on their performance. This, the need for compara-bility and fairness to other system development projects and students at the faculty, issues the necessity of rules. In the following the most essential organizational framework is de-tailed to define the project’s environment. The project is organized as two-week block course from Sep-tember 23rd to October 4th 2013 (“project-weeks”). A two-day “follow-up” meeting is scheduled for 21st-22nd November 2013. Attendance is compulsory for all participants. Three preliminary meetings (June 27th, July 04th, July 18th) enable the students to get familiar with the topic, the motiva-tion, and challenges. Attendance is compulsory. The partici-pants are split into six development teams of at least two and up to four students and tasks are assigned to those teams. It is expected that students prepare for the respective assigned task upfront the project-weeks. Each team must prepare a project report, detailing their ap-proach, made decisions, findings, arising problems, and results using an appropriate presentation.

Each student must present individual achievements on the last day of the project-week (October 4th 2013). The grades are given based on the performance of each indi-vidual student and the overall success to achieve the given tasks.

III. PROJECT MANAGEMENT It is part of the project, to work together on different but inter-dependent parts of the system. This demands for adequate coordination and first of all communication. Even though the emphasis is not placed on managing projects, it is still neces-sary to provide transparency and comprehensiveness for all participants related to the most essential characteristics and developments regarding the different software modules. Therefore “user-stories” recorded on paper cards are used to describe sub-tasks. They support the development process by making tasks and hence the progress and problems visible in a very early development stage (see Table 1, Fig. 3) [PiMi08].

Furthermore, user stories may help the participants to under-stand the tasks and their complexity by documenting depend-encies and priorities. Placed on a wall or board, the “critical path” of the development becomes visible. This means they can also help to setup a system integration plan. Actually, integration of the modules plays a very essential role for such a parallel development approach based on six development teams with a given timeframe of only two weeks, whereas most of the participants are new to the project domain and their implications (automotive, CAN, Android, embedded, etc.). Based on this assumption, the overall development ap-proach is based on short development cycles with integration

Fig. 2. Intel Atom based DE2i-150 FPGA Development Kit.

TABLE I USER STORIES – INFORMATION ON CARD

Name Description Example

ID team-ID + story-ID 4.03 Title 2-6 words, active re-establish

connection after network loss

Duration estimation in days 0.75 Prio weight in relation to achieve

success for the overall pro-ject (high/mid/low)

mid

Dependencies dependencies to other sto-ries/tasks

4.02, 3.04 (this means 4.03 directly depends on 4.02 and 3.04)

Responsible max. two participants Tom & Jerry Start/End when story was implemented 25-sep – 26-sep

Fig. 3. Example of a user story on a paper card according to Table 1.


3

latest on every second day. The progress is visualized by placing the cards into three dis-tinct categories, as depicted in Fig. 4.

IV. SYSTEM ARCHITECTURE Several tasks have to be fulfilled to achieve the project goals, visualized in Fig. 5 by use of a layered architecture. This fig-ure also references the following tasks:

(1) Read data from vehicular fieldbus. (2) Encapsulate data. (3) Send encapsulated data. (4) Receive encapsulated data. (5) Decapsulate data. (6) Process and visualize data.

Based upon this abstracted view on the overall system, further sub-tasks are derived for the application logic processed on the embedded platform (e.g. “CAN-Server”), the communication (or streaming of data), and the “APPs” deployed to the mobile devices. Those also correspond to the fundamental require-ments defined in section I, which have to be interpreted as conceptual framework for the project. The specified system characteristics still leave significant room for interpretation and are target of more detailed specification within the context of the preparation and the project-weeks. Nevertheless, all tasks inherently are to be implemented using C/C++, except for the applications targeted for the mobile devices.

V. DEVELOPMENT TEAMS During the preliminary meetings six development teams were defined, assigned to three different topics:

A. System Team #1: Setup the embedded platform and provide a man-

agement interface to configure the system. Team #2: Gather sensor data using the vehicular CAN bus

and a GPS device.

Team #3: Provide a data container that provides an interface to access the available data using synchronized shared memory structures.

B. Communication Team #4: Serialization and transmission of data, including

socket-server functionality.

C. Mobile Applications Team #5: Receive data from communication socket and pro-

vide interface to ease the processing. Team #6: Visualize information as digital instrument cluster

and process data to calculate the current vehicles’ power “applied to street”.

Each team manages itself, but has to negotiate details of the interfaces with related teams. This applies as well to the inte-gration and test plan.

VI. SYSTEM DEVELOPMENT Based on the requirements and the available embedded plat-form as detailed in section I, the project environment was prepared before the project-weeks. This affected the work-stations within the lab, but foremost the embedded platforms and the respective operating system. The latter is a Linux based operating system, which is built using the Yocto Pro-ject’s toolset [Wi12, YOCTO]. This toolset mainly consists of the Poky build system that relies on openembedded-core and BitBake. This means, with Yocto an extensive Linux distribu-tion can be built, consisting of only the necessary stuff, cus-tomizable to the actual project’s needs. Exactly that was done during preparation of the project CAN2WiFi: Select existing, create and add new components, structured in so-called layers. To the existing layers that cover the specifics of the hardware architecture and the on-board peripherals, a ICM layer is made available that provides drivers and configuration to operate the platform in WiFi access-point mode, connect it to CAN via a USB device and do on-target simulation of a CAN communi-cation. This reduces the necessary efforts during the project-weeks and enable the development teams to build upon a solid and easy reproducible foundation. Due to the parallel development the configuration manage-ment system Perforce is used. In the following, the implementation of the system and the achieved results are described. Therefore the above defined partitioning of the project’s tasks is used.

A. Platform (Team #1) The software modules of the embedded platform are imple-mented as separate applications, to foster a loose coupling by keeping dependencies on the code and build level as small as possible. This also has positive implications to the “Platform-Team”, because the distinct binaries can be started inde-pendently of each other. The system startup is implemented using a bash script containing the following four steps to en-sure a predictable startup based on a clean environment:

(1) Remove all potential leftovers from a previous execu-tion.

Fig. 4. Visualize the development progress using user story cards.

Fig. 5. Layered system architecture.


4

(2) Restart the CAN interface. (3) Remove lock files. (4) Start the “canManagement” executable.

The script is added to the system startup routine, to make sure the embedded platform is up and running after reboot. The “canManagement” is implemented using C++ and calls fork and execvp for each of the following binaries to start the application logic of the embedded platform:

• canReader (Team #2 and Team #3) • network (Team #4 and Team #3) • manage (Team #1)

The “canReader” has to be started first, because it sets up the data container as an interface via shared memory that is uti-lized by “network”. This means, the order during the startup is important. Further, only one instance of “canManagement” is allowed. To ensure this a simple but effective file lock mecha-nism is used. Basically, the “canManagement” could also be implemented using an additional bash script. An implementation in C++ provides more opportunities to enhance the system, e.g. im-plementing a watchdog mechanism. The management of the system is realized by using a com-mand-line interface. The communication between the running applications is implemented using the inotify mechanism. This is event based and therefore prevents unnecessary polling. Adapting the configuration is part of the management. This includes the configuration of the WiFi access-point and covers the values for the SSID, channel, MAC-filter, passphrase, and encryption. The configuration can be saved and restored using an XML formatted text file. An XML reader and writer, based upon the libxml2 library, realize this. Additionally, current system stats are available through the command line based management interface, including e.g. uptime, process list, system load, and information regarding the CAN driver. For remote management, a web interface provides a graphical interface to view system stats and adapt the configuration. The web interface is implemented as system service using C++. This service is registered as xinetd service using port 6553. This means, there is no need for an extensive webserver which may affect the overall system performance.

B. CAN and GPS (Team #2) The “canReader” is the application that gathers the vehicular data from the CAN bus using the PEAK CAN USB device connected to the DE2i 150 FPGA Development Kit and the respective driver abstractions of the Linux kernel. To substitute an actual CAN bus, a virtual driver abstraction is available which can be fed with previously captured CAN traces to ease the development and testing within the lab envi-ronment. Therefore the kernel module “vcan” has to be acti-

vated. Within the ICM labs the simulator “carsim” was devel-oped, which is able to playback CAN traces to the defined interface. This is used to feed the “vcan” device for on-target testing without the need for further hardware. Alternatively another computing device with CAN interface may be used accordingly to simulate the use of the embedded platform within an actual vehicle. This is not necessary since the em-bedded platforms used provide sufficient computational power to host the simulation during the development and early test phase. To achieve the goals of the project, the gathered CAN data needs to be parsed and pre-processed to store it within a shared memory region for further processing. That shared memory region is the “Data Container” described in section VI-C. Additionally, the raw CAN messages are also made available for further processing to enable the user(-applications) to ei-ther make use of a raw-mode or a data-mode with already pre-processed data fields. The CAN data are filtered by use of an effective library that eases the use of CAN socket communica-tion, developed during a previous project at the ICM labs. Although the CAN communication is very strict using 8-Byte messages with a leading identifier, each car manufacturer develops a unique protocol how the data is encoded or struc-tured. Within this project the CAN protocol of a BMW 5 se-ries is used, because this vehicle is available for trial runs. The following CAN-IDs are available which were identified during previous projects:

• Speed • Revolutions per Minute (RPM) • Battery Voltage • Mileage • Reach • Throttle Control • Indicator Lights • Door Status

Further, a GPS device is used to add spatial data to the set of information available through the CAN bus. Therefore an uBlox6 receiver is connected to the USB interface of the em-bedded Platform. A GPS parser based on an already available implementation is used to provide the positioning data using the data container within the shared memory region. The de-rived “gpsReader” is implemented as a separate thread, com-puted independently of the “canReader”-thread. The gyroscope sensor attached to the embedded platform’s FPGA is not used within this project scope. The available documentation and tutorials are extensive, but based on the estimation with regards to a brief evaluation it was decided to postpone an integration of the gyroscope and the FPGA due to lack of experience and resources.

C. Data Container (Team #3) The data container acts as interface for the sensor data be-tween the different software modules. It is incorporated into


5

those software modules, which means it does not appear as a separate thread or process. Basically, it defines the structure of the shared memory region and provides methods that abstract the access to that shared memory. The design of the data con-tainer follows the specification of Team #2, which is responsi-ble for the sensor data. It follows an object oriented design that covers three different types of shared data, all implement-ed by use of a distinct class that is derived from a common abstract SharedData class:

• raw CAN data • parsed CAN data • GPS data

Additionally to the data values, timestamps are stored to miti-gate introduced transmission latencies for further processing within the user application on the mobile device. Based on the fact that the data container acts as an interface between parallel and concurrent computed software modules, the access ensures thread safety via the implementation of the assessors by use of mutual exclusions (e.g. POSIX mutexes). Further, appropriate synchronization functionality is necessary to signal the data consumer (e.g. “network”) when a new value is stored in the shared memory region by a producer (e.g. “canReader” or “gpsReader”). Therefore trigger semaphores are utilized, based on POSIX semaphores [WiTr05].

D. Network Communication (Team #4) Assuming the data is available within the data container, the next building block is to make it available through a network connection. Therefore a protocol is defined for serialization of the vehicular sensor data (e.g. parsed CAN, raw CAN, and GPS) as well as for the control of the data flow (e.g. subscribe, unsubscribe, pause, and resume). The “network” module establishes a server listening for sub-scribing clients. Therefore an already available socket frame-work is used, that eases the management of multiple incoming client connections and the creation of corresponding handler threads. The communication between the threads is based on shared memory, due to performance reasons. The TCP proto-col is used for the communication to make sure no packets get lost. This software module makes use of object oriented sys-tem architecture.

E. Android Network (Team #5) The mobile device connects to the embedded platform using a TCP socket. This is realized by use of a service implementa-tion, which encapsulates the socket communication and pre-vents multiple connections for different instances of an An-droid “Activity”. It is implemented using Java. Within the service, a class “SocketClient” manages the con-nection to the server. It is also responsible for receiving, trans-forming, and redirecting the data to the “Activities”. The “SocketClient” implements the singleton pattern to ensure only one instance is running. Additionally, the socket layer within the Android device pro-vides the capability to send commands to the server to control

the data flow (e.g. subscribe, unsubscribe, pause, and resume). Those control commands are issued by the user application based on input events triggered by the graphical user interface. For the implementation the Android Interface Definition Lan-guage (AIDL) is used to define the interface between the socket layer and user applications. The received data is transformed according to the protocol definition of the embedded platform’s network software mod-ule (Team #4). This means, the data is extracted from a byte array and copied into designated data fields according to the respective CAN identifier for further processing (e.g. visuali-zation or calculation of the vehicle’s actual power). The data layer realizes this. It also stores the incoming data with their timestamps for further interpretation into an SQLite database. This enables the visualization of graphs and the calculation based on data for a given timespan. This data is not persistent.

F. Android User Interface (Team #6) Based on the interface provided by the project’s Android net-work layer, the user interface visualizes the vehicle’s current velocity, the power (based on the acceleration), and all other available data as specified in section VI-B. Therefore an Android Activity with its predefined callback methods is implemented. The user interfaces relies on the Android framework and therefore is implemented using Java. The main view is the digital instrument cluster, which visual-izes the data using nicely rendered gauges on a canvas object, depicted in Fig. 6. Additionally a simplified and more tech-nical view displays the values using digits. Also a graph view is implemented to visualize value changes over time. More prototypes were implemented, which will not be detailed with-in this document.

VII. EVALUATION The simulator in combination with the virtual CAN interface provided good support during the development phase within the lab environment, as detailed in section VI-B. Due to the availability of multiple embedded platforms and several mo-

Fig. 6. Android tablet with digital instrument cluster connected to vehicle .


6

bile Android devices the different development teams were able to proceed with their tasks without the need to compete for accessing the target devices. This had positive effect on the motivation of the participants. Even more positive effect had the availability of a test vehicle, which provided access to the CAN fieldbus and the opportunity to do test trials within a real-world scenario (e.g. not only based on playback CAN traces), as depicted in Fig. 7.

During those life-tests unexpected behavior was observed, due to low-voltage situations during startup or temporary loss of connection regarding the USB CAN interface. Such experi-ence has significant impact on the learning effect, which on balance is the main target of this project. This is also supported by the participants’ answers within the context of the project evaluation performed by the university’s quality assurance, as depicted in Table II.

Furthermore, the open-ended question within this question-naire contained the following eight statements:

“The whole study should consist of such courses” “My review speaks for itself J” “Awesome. Learnt that much within two weeks!!” “(…) very nice course!” “(…) This was fun! And I learnt a lot. The practical experi-ence is worth a mint!!!”

“ Neat project!” “J” “(…) Could hardly await the next project. Our university should do more projects”

VIII. SUMMARY The project was successful. This can be stated for different reasons. First and foremost the participants’ motivation rose during the project-weeks. Whereas most of them didn’t have in depth knowledge about the project domain (e.g. CAN, au-tomotive, embedded, Android), 92.3% stated to have learnt a lot while rating the course with 1.08 within the course evalua-tion. Furthermore, they stayed until very late during the pro-ject-weeks and also improved the system afterwards without compulsion. During presentations and discussions with the participants, the students showed awareness of the difficulties and also their enthusiasm. Additionally, the project environment seemed to have positive effect on the project results. The DE2i 150 FPGA Develop-ment Kits fulfilled exactly the needs. The Yocto based operat-ing system in combination with the enhancements incorpo-rated as part of the project preparation did their job without any problems. Both the participants’ enthusiasm and a proper environment might be the key to the overall success: A functional system rendered by 20 developers partitioned into six independent teams within ten days. Sure, there are several parts of the system that could and should be improved, but the implement-ed prototype fulfills the required tasks and founds a base for future projects.

ACKNOWLEDGMENT We have to thank Intel for the generous providing of embed-ded platforms and the uncomplicated and very helpful support during the preparation of the project, most notably Raluca Oltean and Joy Shetler. Further, we have to thank a number of scientific staff mem-bers, which actively supported the preparation of the platforms and guided the participants during the project-weeks. The project wouldn’t be that successful without their contribution. In alphabetical order:

• David Diefenbach (embedded OS, Yocto, CAN) • Clemens Fischer (Perforce, SW integration, car setup) • Markus Glaab (network, Android) • Tobias Holstein (user interface, Android) • Bettina Kurz-Kalweit (Perforce) • Pierre Schnarz (Android)

REFERENCES [PiMi08] D. Pilone and R. Miles: Head First Software Development.

O’Reilly Media, 2008. [We13] B. Weißbach: Entwicklung und Implementierung eines CAN

Leistungsmesser auf einem Android Gerät. Bachlor’s thesis, h_da Hochschule Darmstadt, 2013.

[Wi12] J. Wietzke: Embedded Technologies - Vom Treiber bis zur Grafik-Anbindung. ser. Xpert.press. Springer, 2012.

[WiTr05] J. Wietzke and M. T. Tran: Automotive Embedded Systeme: Effizientes Framework – Vom Design zur Implementierung. ser. Xpert.press, Springer, 2005.

[Yocto] Yocto Project: Yocto Project Website. A Linux Foundation Collaborative Project, https://www.yoctoproject.org/

Fig. 7. DE2i 150 FPGA connected to test vehicle.

TABLE II RESULTS OF THE COURSE EVALUATION QUESTIONNAIRE

Question Grade (mean) (1-5, 1=good, 5=bad)

Lecturer 1.00

Performance 1.02

Student 1.84

Ensemble 1.08

can2wifi: project report in car multimedia project … · usually limited to audio streaming and...

Documents