assessing the sop of mbe in the embedded systems domain xubo miao msc, school of computing...

Assessing the SoP of MBE in the Embedded Systems Domain

Xubo MiaoMSc, School of ComputingSupervisor: James R. Cordy

1. Motivation

• MBE aims at increasing the effectiveness of engineering by using models as key artifacts in the development process.

• Only little work targeting the embedded systems domain

• The authors did a study on the use and the assessment of the MBE in embedded systems domain

2. Related Work

• Agner et al. present the results of a survey on the use of UML and model-driven approaches in the Brazilian embedded software development industry.

– 45% of the 209 participants use UML. And the majority are experienced developers working at medium-sized companies.

– Models are mainly used for documentation, with only little use of code generation or model-centric approaches in general.

2. Related Work (Cont’d)

• Kirstan and Zimmermann report a case study within the automotive domain.

– Positive Effects: an earlier detection of errors, a

higher degree of automation, and cost savings

during the initial phases of development

–Negative Effects: large function models can

become too complex and that interoperability

between tools is difficult.

3. Two Research Questions

• RQ1: What is the current SoP and the assessment of MBE in the embedded systems domain?

–Modeling environments, modeling languages, types of

notations, purposes that models are used for

• RQ2: How does the use and the assessment of MBE differ between different demographic subgroups in the embedded systems domain?

– E.g. automotive and avionics domain, new and highly

experienced MBE users

4.1 Research Methodology – Study Design

• The survey questionnaire consisted of 24 closed-ended and open-ended questions

– 1st Part: contains 13 questions gathering demographic data

– 2nd Part: contains 11 questions addressed RQ1

– Both 2 parts are considered together for answering RQ2.

One Minute for you to go through the questionnaire

4.1 Research Methodology – Study Design (Cont’d)

• Derived 24 hypotheses from the related work to guide the data analysis for RQ1

• Derived 8 hypotheses from their own view on MBE after designing the questionnaire to answer the RQ2

One Minute for you to go through all hypotheses

4.2 Research Methodology – Data Collection

• The surveys are distributed to partners from five different projects.

• 121 out of 196 surveys are completed.

• 9 surveys are excluded based on degradation points computed from missing answers and the time to fill out each survey page.

• Therefore, 112 answered surveys for data analysis

5.1 Results – RQ1: State of Practice

• Modeling tools and languages

– Tools: most use Matlab/Simulink (50) or Eclipse-based

MBE tools (34)

–Notations: Finite State Machines (74), sequence-based

models (64) and block diagrams (61)

– Functional aspects of a system participants already use models: structure (68), discrete state specifications (48) and static interfaces (47) are most common

5.1 Results – RQ1: State of Practice

• Needs for Introducing MBE

5.1 Results – RQ1: State of Practice

• Purposes of Models

5.1 Results – RQ1: State of Practice

• Positive & Negative Effects of MBE

5.1 Results – RQ1: State of Practice

• Shortcomings of MBE

5.2 Results – RQ2: Differences by Subgroups

• They use Fisher’s exact test (two-tailed) with a level of significance α ≤ 0.05.• H2.1, H2.6, H2.7 - p ≥ 0.05• H2.4 – not enough data for each subgroup in

order to compare the groups• Therefore, only H2.2, H2.3, H2.5, H2.8 are

statistically significant.

5.2 Results – RQ2: H2.2

• Traceability (p = 0.00017), safety (p = 0.018), and reusability (p = 0.019)

• Supporters perceive the effects of MBE on these three aspects significantly more positive than opponents and nuetrals.

5.2 Results – RQ2: H2.3

• Cost (p = 0.016) and traceability (p = 0.006).

• Participants who are still using MBE report more positive effects on cost and traceability than participants who stopped using MBE.

• A possible explanation for the few significant differences might be that participants who stopped using MBE did so because they moved to a different position.

5.2 Results – RQ2: H2.5

• Quality(p = 0.011)

• Participants who reported that many usability issues with tools mostly or fully applies rated the effects on quality slightly less positive than participants who reported that usability issues apply at most partly.

5.2 Results – RQ2: H2.8

• Supporters: 51 of MBE reported to use MBE tools more than non-MBE tools or only MBE tools, and 18 reported to use less MBE tools than non-MBE tools or only MBE tools at all.

• Opponents and Neutrals: 5 and 13

• MBE promoters use more MBE tools in comparison to subjects neutral or opposed to MBE.

6. Conclusion

• MBE is widespread in the embedded domain.• Models are clearly not only used for informative

and documentation purposes, but also used for simulation, code generation, test case generation, traceability and timing analysis.• The survey respondents reported mostly positive

effects of Model-Based Engineering