a systematic mapping study on high-level language virtual...
Post on 17-Jun-2020
3 Views
Preview:
TRANSCRIPT
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
A Systematic Mapping Study on High-levelLanguage Virtual Machines
Vinicius H. S. Durelli, Katia R. Felizardo, andMarcio E. Delamaro
Computer Systems DepartmentUniversity of Sao Paulo (ICMC-USP)13560-970 – Sao Carlos – SP – Brazil{durelli,katiarf,delamaro}@icmc.usp.br
October 17, 2010
1 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
Agenda
1 MotivationResearch on HLL VMsFirst Step Towards Filling in Such a Gap
2 Systematic MappingOverviewStepsData Extraction and Mapping
3 Threats to Validity
4 Concluding Remarks
2 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
Research on HLL VMsFirst Step Towards Filling in Such a Gap
Research on High-level Language Virtual Machines
A great deal of the contemporary high-level languages havetheir execution environment based upon high-level languagevirtual machines (HLL VMs).
There is a large body of literature on research in virtualmachine for high-level languages.
A mature research area means a sharp increasein the number of results made available, thus itbecomes essential to summarize and provide anoverview of such area.
To the best of our knowledge there are no comprehensivestudies focusing on an overview of this research area andits most investigated subjects.
3 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
Research on HLL VMsFirst Step Towards Filling in Such a Gap
Motivation: First Step Towards Filling in Such Gap
In order to fill in such a gap it is needed to ascertain thenature, extent, and quantity of published research papers.
Contributions:
1 Areas that have been most subjected to investigation.
Side effect: Areas that require further research.
2 The relevant publication forums.
3 HLL VM implementations that are the most widely usedwithin the academic community.
4 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Evidence-based Paradigm
Definition → Systematic Mapping
Methodology that involves searching the literature in order to aggre-gate and categorize primary studies, thereby yielding a synthesizedview of the research area under consideration [Petersen et al., 2008].
Advantages:
The approach used for searching and inclusion and exclusioncriteria are defined in a research protocol and reported asan outcome.
Side effects: Transparent;Replicable;Updatable.
5 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Systematic Mapping Process: Overview
Figure: The systematic mapping process [Petersen et al., 2008].
6 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Research Questions
Research questions must embody the mapping study purpose.
RQ1: which functionalities/features/characteristics of HLLVMs have been most investigated?
RQ2: which are the mainstream HLL VM implementationswithin the academic community?
7 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Search for Primary Studies
Search String → combination of these keywords and acronyms
virtual machine, VM, high-level language virtual machine, and HLLVM.
We used the search string on the following electronic databases:
ACM Digital Library,
EngineeringVillage,
IEEE Xplore,
Springer Lecture Notes in Computer Science (LNCS), and
ScienceDirect.
No limits were placed on date of publication.
8 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Screening: Inclusion Criteria
The inclusion criteria devised and applied are:
if several papers reported similar studies, only the mostrecent was selected;
papers describing more than one study had each studyindividually evaluated;
it has to describe at least a prototypical implementation ofthe proposed improvement, thereby mentioning the HLLVM implementation that was modified.
9 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Screening: Exclusion Criteria (i)
and the following exclusion criteria:
papers that do not present studies pertaining to HLL VMs,e.g., papers describing research on system VMs;
studies describing the introduction of improvements thatconsist in solely modifying the intermediate language of theHLL VM under consideration;
studies whose proposed enhancements do not imply inmaking changes to the underlying HLL VM, e.g., papersdescribing features implemented atop HLL VMs;
10 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Screening: Exclusion Criteria (ii)
studies whose target HLL VM is either a co-designed (e.g.,composed of both software and hardware portions) or anentirely implemented in hardware HLL VM;
technical reports, documents that are available in the formof either abstracts or presentations (i.e., elements of “grey”literature), and secondary literature reviews (i.e., mappingstudies).
11 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Final Set of Selected Primary Studies
Electronic Database Number
ACM Digital Library 1554
EngineeringVillage 1395
IEEE Xplore 309
Springer LNCS 640
ScienceDirect 1123
Total 5021
Candidates 142
Final set 128
Table: Papers retrieved from each electronic database, total of candidatestudies, and the final set.
12 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Keywording
The aim of this step is to devise our own classificationscheme and categories for the selected primary studies.
Certain sections are read for the purpose of findingkeywords and concepts that reflect their contribution.
13 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Resulting Categories
CategoriesOptimizationGarbage Collection (GC)DebuggingMemory Leak Tolerance (MLT)New Language Construct (NLC)ProfilingAspect-Oriented Programming (AOP)Embedded System (ES)SecurityReal-TimeDistributed Computing (DC)Fault Tolerance (FT)Resource Sharing among HLL VMs (RSVM)Testing
14 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Resulting Category Frequencies
Testing
RSVM MLT
Security NLC FT DC
DebuggingAOP
Profiling
Real-Time ES GC
Optimizatio
n
Category
0
5
10
15
20
25
30
35
Fre
qu
en
cy
2 2 23
4 4
78 8
9
13
24
3334
Figure: Frequency of studies in each category*.
∗Certain studies were grouped in more than one category
15 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Most Researched Subjects Evolution
According to our results, these are the “trendy” subjects:
19981999
20002001
20022003
20042005
20062007
20082009
Year
0
1
2
3
4
5
6
7
Fre
qu
en
cy
OptimizationGCES
Figure: Year-wise distribution of publications on the most investigatedcategories.
16 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Distribution of Primary Studies by Electronic Database
EngineeringVillage
30.0%
ACMDigital
Library
48.0%
Springer LNCS
13.0%
IEEE Xplore
9.0%
Electronic Databases
Electronic Database Number
ACM Digital Library 62
EngineeringVillage 38
Springer LNCS 16
IEEE Xplore 12
ScienceDirect 0
17 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Distribution of Primary Studies by Publication Type
Conference
36.0%
Journal24.0%
Symposium
20.0%
Book Chapter
14.0%
Workshop6.0%
Publication Type
Publication Type Number
Conference 46
Journal 31
Symposium 25
Book Chapter 18
Workshop 8
18 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Map: Year-Wise Distribution (detailed)
1999
1998
2002
2003
2000
2001
2006
2007
2004
2005
2008
2009
2010
FT
NLC
DC
Optimization
GC
MLT
Real-Time
Security
ES
Debugging
RSVM
Profiling
Testing
AOP
1
1
1
2
3
1
2
1
1
4
2
1
1
1
1
3
1
1
2
1
1
2
1
2
1
4
5
1
4
3
4
2
7
1
1
1
1
3
3
1
5
2
5
2
1
1
1
3
2
2
6
4
1
3
2
6
1
1
2
3
1
2
1
3
3
4
1
1
1
1
1
1
1
Map
19 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
OverviewStepsData Extraction and Mapping
Map: The Most-Widely Used HLL VM Implementations
GForth
VM*
Sabl
eVM
CVM
Har
mon
y
Sim
pleR
TJ
OCV
M
VMKit
Jikes
RVM
IBM
's J2
ME
MONO
Hot
Spot
JESS
ICA2
CACA
O
TclV
M
JeRTy
Stea
mlo
omKaf
feCL
R
Exac
t VM
Jam
aica
Max
ine
JITSJ9
IVM
CEJV
MKVM
ORP
SICS
tus
Jam
VMOVM
FT
NLC
DC
Optimization
GC
MLT
Real-Time
Security
ES
Debugging
RSVM
Profiling
Testing
AOP
12111 6 1 332 3 1
421 1
11 1 52
2 13 1
1 1 1
1 1
31 11 2
123 4 1 2 1 13 11 111
2 1 2 2 13
5 52 3 1 2 1 11 1
2
11 11
1 12
2
Map
20 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
Threats to Validity
We cannot rule out threats from a quality assessmentperspective.
(We wanted to be as inclusive as possible) We simplyselected studies without assigning any scores.
Another threat consists in whether we have properlyidentified and selected all relevant publications.
Whether our resulting classification scheme and cate-gories are coherent also represents a threat to validity.
21 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
Concluding Remarks
The mapping study results, although not entirely surprising (somemay argue), can be used to support several claims that arefrequently made but not scientifically backed up.
Our mapping study reveals that the majority of researchinto HLL VMs focuses on optimizing these executionenvironments, improving their memory management capa-bilities, and tailoring them to resource-constrained settings.
As for the publication types, the majority of the studies areconference publications.
Another contribution of this paper is the map we havecreated.
22 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
References
K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson.Systematic Mapping Studies in Software Engineering.12th International Conference on Evaluation and Assessmentin Software Engineering (EASE), pages 71–80, 2008
J. E. Smith and R. NairThe Architecture of Virtual Machines.Computer 38(5):32–38, 2005.
J. E. Smith and R. NairVirtual Machines: Versatile Platforms for Systems andProcesses.Morgan Kaufmann, 656 pages, 2005.
23 / 24
MotivationSystematic Mapping
Threats to ValidityConcluding Remarks
A Systematic Mapping Study on High-levelLanguage Virtual Machines
Vinicius H. S. Durelli, Katia R. Felizardo, andMarcio E. Delamaro
Computer Systems DepartmentUniversity of Sao Paulo (ICMC-USP)13560-970 – Sao Carlos – SP – Brazil{durelli,katiarf,delamaro}@icmc.usp.br
October 17, 2010
24 / 24
top related