![Page 1: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/1.jpg)
A survey on software quality practice -Pilot study in the Walloon region
Javier Perez, Tom Mens, Service de Génie Logiciel, Université de Mons
Flora Kamseu, Naji Habra, PRECISE lab, FUNDP
Presented at SATToSE seminarUniversity of Koblenz, August 2012
SATTOSE 2012
![Page 2: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/2.jpg)
Context
• Portefeuille TIC – ERDF project lead by CETIC (2007-2013)
• CEIQS: Center of expertise in engineering and quality of systems
– aimed at developing a portfolio of innovative techniques allowing local companies to master the diversity, complexity, quality and rapid evolution of information systems
• workpackage QUALGEN– collaboration between FUNDP and UMONS since 2010
– Supported by Wallonia
2SATTOSE 2012
![Page 3: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/3.jpg)
3SATTOSE 2012
Objectives
• Explore how quality-related software development practice is being performed in industry
• Compare this across different regions and countries
• Relates this to what is being taught in academia– Is there a gap between teaching and industry
needs?– What good methodologies, practices, tools are
not being used and why?
![Page 4: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/4.jpg)
About the survey
• Online survey carried out in Walloon region– Using LimeSurvey, from 29/5 till 30/6 2012– Companies involved in software development or
software maintenance
• Addressed topics– Use of processes during software development and
maintenance– Use of software quality measurement and improvement– Use of quality models and quality standards– Use of testing– Organisational support of development teams
4SATTOSE 2012
![Page 5: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/5.jpg)
Structure of the questionnaire
• Introductory questions (6)– Details of respondent and company
• General development questions (5)– Perspective on dev. practices carried out by company
• Structural software quality (5)• Software testing and maintenance (5)• Quality models and quality standards (5)• Organisational support of development teams (5)
5SATTOSE 2012
![Page 6: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/6.jpg)
Respondents – Number
– Initial mailing sent out to 145 companies
– 71 responses out of 188 contacted respondents• Response rate 37,8%
– Responses from 47 different companies• Multiple responses from same companies were
aggregated into a single one
– Incomplete responses were ignored
– 44 fully completed questionnaires kept for analysis• Corresponds to 62% of received responses
6SATTOSE 2012
![Page 7: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/7.jpg)
Respondents - Origin
7SATTOSE 2012
Mainly from Wallonia and Brussels (the target of our pilot study)
![Page 8: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/8.jpg)
Respondents – Company size
8SATTOSE 2012
Good balance between company size (number of employees) of respondents
![Page 9: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/9.jpg)
Respondents – Training Level
9SATTOSE 2012
Many at master+engineer level (18+17) and bachelor (20) levelFew or none with PhD (10+26), other degree (10+16) or no degree (3+27)
![Page 10: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/10.jpg)
Process – Dev. process support
• Use of a well-defined and well-documented development process? 45,5%– No: 19– Yes: 20 (agile or scrum, Prince 2, RUP, ISO
certification, proprietary)– Don’t know: 5
• Use of agile practices or methods? 63,6%– No: 12– Yes: 28 (17 mention SCRUM, 2 mention Prince2)– Don’t know: 4
10SATTOSE 2012
![Page 11: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/11.jpg)
Process - change or configuration management process
11SATTOSE 2012
Use of change or configuration management process is highly popular 73,8% (31/42)
![Page 12: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/12.jpg)
Process - perceived importance factors for software project success
12SATTOSE 2012
• Rated from (1) not important to (5) essential
Average : 4 4,2
![Page 13: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/13.jpg)
Process - Creation and modification of software artefacts
14SATTOSE 2012
Are arch. descr. (36,4%) and design models (39,5%) being evolved?
![Page 14: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/14.jpg)
Tools - Use of integrated platform for
15SATTOSE 2012
Version control (97,6%) and bug tracking (92,7%) well establishedPlatforms for continuous integration (57,1%), configuration (55,6%) and testing (60,5%) a bit less
![Page 15: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/15.jpg)
Tools - Programming languages used
16SATTOSE 2012
OO languages most popular (Java, C#, C++), followed by scripting languages. SQL and Cobol legacy also remains important.
![Page 16: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/16.jpg)
Tools - Development environments used
17SATTOSE 2012
Dichotomy between Java and .Net visible at IDE level.Many others but much less frequently used.
![Page 17: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/17.jpg)
Tools - support for design models, documentation and code synchro• Very basic
– Visio (10), Word (8), Sparx Enterprise Architect (5), Doxygen (3), StarUML (2), Confluence (2) and many others
– Little use of UML modeling tools
• Little or no support for model-code synchronisation
18SATTOSE 2012
![Page 18: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/18.jpg)
Quality - Use of design patterns
19SATTOSE 2012
Use of design patterns is highly popular 72,7% (32/44)
![Page 19: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/19.jpg)
Quality - Use of qualityimprovement techniques
20SATTOSE 2012
Often or continuously
36,6% (15/41)
42,9% (18/42)
35,7% (15/42)
58,5% (24/41)
Only moderately popular (35,7%<x<43%) except for refactoring (58,5%)
![Page 20: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/20.jpg)
Use of quality support continued
21SATTOSE 2012
Poor support for quality (no quality tools, processes or models)
36,8% (14/38)
32,4% (12/37)
19,4% (7/36)
![Page 21: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/21.jpg)
Popularity of quality improvement techniques
• Most popular Version control 97,6% and bug tracking 92,7% platforms Change and configuration management: 73,8% Design patterns: 72,7% Refactoring: 58,5%
• Less popular Design improvement (e.g. code smell reduction): 42,9% Bad quality detection tools: 36,8% Metrics and visualisation tools: 35,7% Dynamic analysis tools (profiling etc.): 36,6% Quality support or improvement process: 32,4%
• Unpopular Use of a quality model: 19,4%
22SATTOSE 2012
![Page 22: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/22.jpg)
Reuse of libraries, components and platforms
23SATTOSE 2012
Reuse is highly successful 86% from own company 75% from open source 60% from other companies (excluding “don’t know” results)
![Page 23: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/23.jpg)
Testing - Use of test process
24SATTOSE 2012
Testing is done by nearly all respondents (97,7%), buttest process used by only 46,5% of the respondents
![Page 24: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/24.jpg)
Testing – development time spent
25SATTOSE 2012
![Page 25: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/25.jpg)
Testing - tool support
• Very varied– Mostly unit testing frameworks (8)– Others: Mantis (4), HP Quality Center (4),
Selenium (3), Hudson (2), Jenkins (2), Quick Test Pro (2), and many more
26SATTOSE 2012
![Page 26: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/26.jpg)
Preliminary conclusions
• Wide range of respondents, from very small to big companies, using many different programming languages and development environments
• Strong points– All respondents believe that quality assurance and
testing are very important for project success– Wide use of testing (97,7%), agile practices (63,6%),
design patterns (72,9%) and refactoring (56,5%)– High level of reuse of components/libraries/platforms
(60% to 86% depending on source of reuse)
27SATTOSE 2012
![Page 27: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/27.jpg)
Preliminary conclusions
• Weak points– Mitigated success of processes
• development processes (45,5%), test processes (46,5%), quality support/improvement process (32,4%), quality models (19,4%)
• Exception: change management process (73,8%)
28SATTOSE 2012
![Page 28: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/28.jpg)
Preliminary conclusions
• Weak points– Mitigated success of static and dynamic code analysis
tools for detecting quality issues, visualisation, computing metrics, profiling, etc… (popularity between 35% and 43%)
– Does not reflect the highperceived importance ofquality assurance forsoftware project success
29SATTOSE 2012
![Page 29: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/29.jpg)
What’s next?• Analyse results in more detail
– Correlate results to company size and training level• Are bigger companies more process-driven and smaller
ones more agile? Does training level play a role?
• Report on the results• Repeat the study in other countries
– Identify regional or national trends– Compare differences and commonalities– We need your help here!
30SATTOSE 2012
studies
![Page 30: A survey on software quality practice - Pilot study in the Walloon region](https://reader035.vdocument.in/reader035/viewer/2022062418/553b9b9f4a7959eb2d8b46cc/html5/thumbnails/30.jpg)
Collaborative study
• Research 2.0– Carry out follow-up / more narrow studies– Carry out this study as a collaborative
community effort?
• Store / share / reuse our data with others– Which format/platform/… to use?
• Have a working session on this topic during SATTOSE/SOTESOLA?
31SATTOSE 2012