Software Tools for Low-Level Software andOperating Systems Classes

Maxwell WalterTechnical University of Denmark

Email: maxw@dtu.dk

Sven KarlssonTechnical University of Denmark

Email: svea@dtu.dk

Abstract—Low-level software is a particularly challengingtopic to learn. In this paper, we present a framework and anapplication that makes the task of understanding system levelsoftware and hardware simpler. Traditional analysis tools, suchas profilers and debuggers, are either too difficult to use in thesesituations, or are simply not available.

Hardware interactions, which are typical of low level software,make the challenge of understanding even more complicated. Thetools we have built provide a view into the workings of suchsystems in both a simulation environment and on real hardware.The motivation for our framework, and the visualization toolsthat use it, is the teaching of an undergraduate level operatingsystems course. We discovered while teaching this course thatmany students struggled with the low-level environment andconcepts of the course. We decided then to develop a set ofapplications and utilities to help students understand the innerworking of the systems they were dealing with.

I. INTRODUCTION

Due to the changing nature of the computing landscape,computer science curriculum has been changing in recentyears as introductory classes have focused more on high levellanguages and technologies. Despite this shift in curriculumfocus, there is still a need for courses that focus on low-levelsoftware and hardware/software interactions. Such softwaresystems would include operating system kernels, hypervisors,bare-metal applications, and software support for hardwareaccelerators. This is because, recent trends in both hardwareand software are making low-level software classes moreimportant. These trends such as the Internet of Things, the pro-liferation of custom hardware accelerators, the rise of many-core systems, and FPGA fabric integrated onto processordies, has caused a resurgence of interest in operating systemsdevelopment and research. This, in turn, has rekindled a needfor classes in operating system programming as well as low-level software design.

In teaching a class like the type described above, wherestudents develop their own rudimentary operating system ker-nel through a progressing series of programming assignments,we witnessed the challenges the students were faced with.Students were unfamiliar with command line based tools,due to only having taken previous classes using IDEs. Stu-dents struggled with important C concepts such as memoryallocation and pointers as previous classes were taught inhigher level languages such as Java or Python. This resultedin students having great difficulty understanding the internal

workings of the software as they became bogged down bymore fundamental concerns.

In addition to the difficulty posed by the opaque natureof low-level software, the interaction of such systems withhardware also pose unique challenges. In introductory levelclasses such as this students are not yet familiar with evensimple architectural concepts such as interrupts and registers,the distinction between physical and virtual memory, or evenhow a microprocessor boots and initializes itself and executescode. As a result, teaching students about operating systemsnecessitates teaching simple architectural concepts, as the twoare very inter-related.

To help solve these problems, and provide students with amore advanced platform on which to learn, we have attemptedto provide the students in our class with the more advancedtools that they are used to, such as IDEs and profilers.However, we have discovered that many tools require accessto information that is difficult if not impossible to obtain froman arbitrary running kernel. The frameworks, such as ptrace,that provide this information do not exist, and would be toocomplicated to create, for our simple kernel. This prompted usto develop our own framework, and tools using it, to supportstudents.

Even then, this only provides a view into the operation ofthe system software state. The most important aspect of low-level software is how it interacts with the underlying hardwarearchitecture, which is fundamentally different from user-levelsoftware. In order for students to gain insight into whattheir software is doing they also need information on thosewhat the hardware is doing. This includes things like pagetables, interrupts, traps, and signals, and access to physicalmemory. Our framework must also include ways to accessthis information in order to obtain the information we need.

In this paper, we present two versions of our framework. Thefirst version we developed for use in a virtual execution en-vironment, attempting to maintain architecture and executionenvironment independence. We then modified the frameworkto run on actual hardware, which required surprisingly fewchanges.

Our software framework was originally developed for usein a virtual execution environment, as virtual execution pro-vides complete access to the underlying system and hardwareinternals. The framework sits externally to our simple kernel,and provides access to the information we need in an easy

way. We make use of a technique called introspection, whichallows us to read the state of the data and variables thatwe are interested in [1], [2]. We have also modified severalexecution environments in order to expose useful aspects ofthe underling architecture. Additionally, we can make use ofHDL simulation environments, such as GHDL, combined withvirtualization to incorporate FPGA hardware effects into thesimulation.

While our software framework was designed to run insimulation by leveraging virtual execution environments forits operation, similar approaches can also be used on actualhardware. Our second iteration of our framework is for in-specting real hardware state. The hardware platform that wetarget is microprocessors with embedded FPGA fabric, suchas the Xilinx Zynq. The Zynq has proven to be a popularplatform in education as it pairs a common ARM platformwith reconfigurable hardware [3]. On hardware, we make useof JTAG hardware combined with a small FPGA logic core toprovide a subset of the level of access and resolution we havein our virtualized system.

Our goal with our framework is to provide students witha way to view internal state of their system, both softwareand hardware, without having to add complex trace points ordeal with initialization of hardware such as USART or ethernetPHYs. We also want to provide students with a way to examinethe state of both hardware and software at the same time, inthe same way.

II. MOTIVATION

Our work is due to a trend we noticed while teaching anundergraduate level operating systems class. The entry levelclasses, which we rely on to supply the background studentsneed, had changed. They have been switching to high levellanguages which means concepts such as memory, pointers,and even header files have become unfamiliar to students.The entry level classes also use more advanced IDEs, suchas Eclipse, meaning students are not familiar with commandline operations, or possibly even what a compiler is.

Additionally, students have little to no knowledge of theunderlying complexities of computer hardware. Operatingsystems require fairly intimate knowledge of the underlyingarchitecture and so must be addressed the class. This, then,means the class has to cover a very large amount of diversematerial.

As a result, we found that we need to either scale back thecontent of the class, or provide a more advanced environmentto mitigate some of these issues. Scaling back the class contentto some extent is required, as allowances must be made sothat students can learn important C concepts such as memoryallocation and pointers, and hardware concepts like interruptsand hardware memory management.

More difficult though is what to do about the lack ofknowledge of the environment. Simply taking class time toexplain what a shell is, how to navigate a directory, or howto use a command line debugger is not worthwhile as theyare not contributing to the course learning objectives. We have

instead decided to try and give students access to tools they arefamiliar with which will make working with the assignmentseasier.

We envision using and developing high level tools thatstudents can use to gain an insight into how the software theyare developing in the class works. Familiar tools like IDEs,profilers, and tracing software can help students gain valuableinsight and intuitive understanding of how their softwareworks. These tools will also, unlike their userspace basedcounterparts, provide insight into the underlying hardwareby exposing interrupts, exceptions, and other architecturalfeatures.

A. Incorporating an IDE

The first series of assignment students are tasked with is tocreate a simple operating system. We provide basic boot codeand students fill in the rest as the labs progress. This includesdrawing text to the screen via hardware frame buffers, im-plementing a userspace and context switches, creating systemcalls, and finally creating multiple processes that interact viathose system calls. We used the bochs emulator to run thesoftware the students developed, which is convenient for thispurpose as it has a built-in gdb server that can be used fordebugging purposes.

Our first action was to provide students with access to anIDE, namely Eclipse, so that they have a familiar environmentto work in. We created a project in Eclipse that was ableto build the software and generate a bootable image, allfrom within the IDE. We also created a debug configurationwithin Eclipse that, when started, would launch the Bochsemulator with the bootable image that was built. Additionally,we created a gdb startup script that would connect to the gdbserver in the running bochs instance. This allowed students torun and debug their operating system code, running in bochs,from within the Eclipse IDE.

This setup greatly enhanced the student’s ability to immedi-ately participate in the software lab activities. They were ableto work in a familiar environment, as the Eclipse platform wasused in previous classes. They also did not have to learn thesomewhat esoteric interface of a command line debugger.

We did have one problem with Eclipse, namely that it wasnot designed with such a use case in mind. This caused oddissues, such as the debugger stopping when source files wereviewed outside of the debugging context. These issues werefrustrating for students and demonstrated the need to toolsspecialized for this purpose.

Our Eclipse integration was also unable to provide us withextra insight into the hardware of the execution environment.We were forced by the setup to use a generic gdb targetand the gdb server exposed by bochs. Neither of these toolsare aware of any of the architectural features exposed by thevirtual environment. As a result, if we wanted to add tracingof interrupts and traps, for instance, we could not do so usingthis method.

Fig. 1. The architecture of our framework.

B. Extension to Other Tools

Due to our success with Eclipse, and after discovering itslimitations, we decided to see what other tools we couldincorporate into the class to make the laboratory exercisesmore understandable. Of particular interest were profiling toolssuch as valgrind or a tracing framework like DTrace or sysdig.These tools require a working userspace to function properlywhich meant we were not able to use them directly, howeverwe took inspiration from them in order to create a frameworkfrom which we could create our own tools.

One of the most important aspects of the profiling andtracing tools is their ability to tap into certain parts of thesystem and extract information from them as the system isrunning. We sought to provide the same level of functionalitywith our framework, but without the need to have a workinguser space. Our first iteration of the framework was based ona virtual execution environment.

III. ANALYSIS FRAMEWORK

A pictorial representation of our virtual execution frame-work can be seen in figure 1, and it can be thought of in termsof three separate components. The first is a management andmonitoring interface which provides a way for applications tointerface with the execution environments. The second com-ponent is the execution environment itself that the softwareruns in. The final component is the system interface whichprovides the mechanism by which applications interact withthe software that is running in the execution environment, andwith the hardware that is being virtualized.

A. Execution Environment

The Execution Environment is the portion of the frameworkin which the software that is being examined runs. We havedesigned the framework to be largely agnostic to what sortof environment the software is actually running in. So inreality the execution environment is the interface to thatenvironment, be it a virtualization platform, and emulator orreal hardware. This allows us to use any of a number of

different environments within our framework. Some environ-ments we have investigated include virtualization platformslike QEMU, simulators such as Bochs and gem5, as wellas actual hardware. This allows us to use different platformsdepending on the needs of the class or assignment, while stillproviding instructors with a consistent interface on which tobuild their tools and assignments.

It is important that the framework abstract as much ofthe execution environment away as possible to provide aplatform agnostic solution. There are many different types ofassignments appropriate to different types of environments.Linux programming experiments can use QEMU, experimentson different architectures, such as our internally developedTinuso, can use gem5, while hands-on assignments can beperformed using real hardware. Flexibility is a useful assetwhen designing labs and courses.

B. Management Interface

The Management Interface provides an application withthe ability to discover and control execution environments.Its purpose it to provide a consistent and convenient way toperform a number of necessary actions, including:

• Create/Discover new execution environments,which we will interchangeably call domains. The exactmechanism and capabilities of a domain depend on thecapabilities of the execution environment in question.Virtualization platforms and simulators can be createdwith differing parameters such as memory size, CPUcount and even different architectures. Real hardwareis limited in its setup and can only be discovered, notcreated. Multiple domains can be created or openedat the same time, which allows for different types ofsystem interactions, such as network communication ordistributed filesystems.

• Start, stop, and restart the domains that havebeen created. These are the basic functions that arerequired to manage domains. These functions also allowfor simulating things like random power loss or failureevents, and provide a way to restart the system in thecase of catastrophic failure, which can be quite commonwhen initially developing kernel code. Providing an easyway for students to reset their activities can alleviate thestress of making mistakes.

• Pause the domain. The ability to pause the domain iscritically important for analysis. Pausing gives applica-tions using the framework time to sample and inspectthe system state in a more detailed manner. Withoutpausing, we may read an inconsistent internal state, assome part of it may have changed between when westarted reading and when we finish. Pausing also gives anapplication the ability to modify the internal state of thekernel or software without having to worry about timingor synchronization issues.

C. System Interface

The System Interface is responsible for providing com-munication between applications utilizing the framework andthe domains, in order to collect state information and reporton system status. We categorize the communication with therunning software into two broad types: implicit communicationand explicit communication.

Implicit communication is where the application using theframework receives information from the domain about thestate of the executing software without its knowledge orexplicit action. One primary example of this is the ability of thesoftware to read the contents of physical and virtual memoryfrom any of the domains, through memory introspection. Thesoftware is not aware that this is occurring and is unable toperform any action in response. Other types of information thatwe are able to receive implicitly can include hardware eventssuch as exceptions or traps, and operating system events suchas system calls and user-space/kernel-space transitions. Wecan also extract information such as page tables and hardwareperformance counters.

Explicit communication is traditional program communi-cation where the software intentionally sends a message tothe external framework. This can be used by the softwareto indicate to the framework that a specific code path hasbeen reached or a specific known event has occurred. It canalso be used to transfer a large amount of state informationto external applications. Data structures may be distributedacross memory which could make reading them via memoryintrospection complicated, as multiple pointers must be trackeddown and followed. Also, because it is initiated by the soft-ware running in the domain with its full knowledge, explicitcommunication can also be used without having to pause it,as the synchronization issues will be taken care of.

IV. VIRTUALIZATION IMPLEMENTATION

Our framework is designed to be execution environmentagnostic, however our first implementation was designedspecifically with virtualization in mind. The interface we use toaccess the individual domains is not specific to any one virtualmachine implementation. Currently we have implemented aninterface for QEMU, which is a virtualization and emulationplatform [4]. QEMU is able to simulate a number of architec-tures of interest including x86, AMD64, armhf, and AArch64,which makes it a convenient platform for education. It alsosupports multiple processing cores, and a number of projectsare using QEMU for advanced system simulation that wecould take advantage of [5], [6], [7]. We have also investigatedusing the gem5 simulator with our own architecture, Tinuso,and the bochs system emulator.

QEMU already provides read access to physical and virtualmemory, but we required more information than what QEMUprovides to track system state. In addition to reading memory,we added the ability to modify large portions of memory atonce, which gives us the ability to alter the system state.This also provides us with a crude way to send messages tothe executing software. Additionally, we modified QEMU to

Fig. 2. Communication methods available within the framework.

enable it to send simple messages to the framework in responseto various system events.

Our modified QEMU is able to send messages on hardwareand software interrupts and upon execution of the syscalland sysret instructions. We also included the ability to sendmessages in response to the specific x86 assembly instruction,xchg, when both operands are the same 8-bit register. Thisis a technique used in simulators like bochs to trigger asimulator breakpoint [8], which we make use of to triggerframework events. These modifications were sufficient for ourassignments, but we should be able to expose any architecturalfeature of the virtualization software.

While the framework was designed to be as generic aspossible to different architectures, tying specific methods ofcommunication to the specific assembly instructions restrictsthe implementation to the x86 architecture. The reason ourvirtualization implementation is based on the x86 architecturebecause our previous programming assignments were alsobased on x86. There is nothing in our implementation specificto x86, however. For example, we can also modify QEMU tosend messages on ARM swi and svc system call instructions.

A. Framework Communication

Figure 2 shows the four methods of communication inour framework that can be used by the domains to provideinformation to the monitoring application.

Virtual Hardware Communication: Virtual hardware com-munication is an explicit communication method where theexecuting system communicates with the framework via hard-ware provided by the virtual execution environment. Virtualhardware peripherals such as network cards and serial portscan provide bidirectional communication in the standard way.In addition, many virtual environments provide hardware pass-through mechanisms, including USB and PCI, giving thevirtual execution environment access to real host hardware.

Using virtual hardware teaches students how to interact withhardware peripherals. Dealing with special registers, interruptdriven communication, and memory mapped I/O help studentsto understand that they are working with real and complexhardware behind the scenes. This does add complexity to theinitial software though, as peripheral initialization code willthen be required.

Memory Introspection: One of the most important methodsan application has to obtain information from the domain ismemory introspection. Memory introspection is an implicitcommunication mechanism that allows the application usingthe framework to read from and, in some cases, write toany address in physical or virtual memory. This gives theapplication the ability to sample any data structure in theexecuting software. Consistency is a concern for reading largestructures and avoiding torn writes, but this can be mitigatedthrough multi-sampling for slow changing data structures, orpausing for quickly-changing ones.

xchg Instruction: In addition to the above communicationmethods, it is convenient to have an explicit way to sendlightweight messages from the executing software to frame-work applications. One reason for this is to enable commu-nication very early in the boot process, before hardware andmemory have been properly initialized. We can also use theselightweight messages to send information, such as addresses orshort status information, to framework applications for displayor further analysis.

While the ability to send simple messages is quite useful,an even more useful aspect of the xchg instruction is that weare able to pause the virtual machine in response to it. Somekernel data changes very quickly, making them very difficultto sample. Also, stack data structures can be difficult to locateexternally from debug symbols, and can disappear quickly asthe stack unwinds. Pausing the virtual machine in response toa specific event gives the framework an opportunity to fullysample the data it needs without having to worry about itchanging or becoming stale.

Hardware Events: There are many hardware events gener-ated by low-level software that are of interest when trying tounderstand how systems operate. Events and interrupts, systemcalls, and user-space/kernel-space transitions are crucially im-portant to understanding how a system works and being ableto follow the flow of execution. Using the xchg instruction tosend a message on each of these events would be possible, butwould require a large number of alterations to performancecritical code. Instead, we chose to modify QEMU to sendevents to the monitor on all hardware and software interrupts,exceptions, and system call routines. Using these messagesrequires no alterations to any system code, and can be used incases where the software could be unreliable. For example, inthe presence of a double fault, explicit communication fromthe software may not be trustable as it may no longer beexecuting properly.

As with the xchg instruction events, the virtual machinecan be configured to pause on a particular type of hardwareevent. This is especially useful for hardware events because

it provides applications using the framework time to capturethe state of the system in the face of critical errors. Withoutpausing, the kernel may destroy its state, or the system mayreset before enough information can be obtained for analysis.Critical errors are highly expected when developing kernelcode and this is a scenario that should be considered common.

V. APPLICATION TO HARDWARE

Running in a virtualized environment has many advantagesfor educational purposes. Virtual machines can be created anddistributed to students to ensure that every student has thesame platform to work from. Virtual execution environmentsprovide a view and a level of introspection that is impossibleto obtain in any other way. Virtual environments also don’trequire additional expensive hardware to make use of whereasreal hardware requires additional input/output devices and po-tentially more esoteric devices like programmers and hardwaredebuggers.

Despite these advantages, running on real hardware is crit-ically important for learning low-level software and operatingsystems. Virtual environments, regardless of how accurate theyare, are still just a software implementation and can differ inimportant ways. Virtual environments are often more forgiv-ing, and the initialization of the various hardware componentswill differ, sometimes significantly. For these reasons wehave taken the ideas from our software framework, originallydeveloped with virtual environments in mind, and applied toreal hardware.

The platform we have chosen for our hardware implemen-tation is the Xilinx Zynq, which combines a ARM Cortex-A9with a Xilinx FPGA, providing both software and hardwareprogrammability. The FPGA is attached to the memory busof the ARM processor and hardware developed in the FPGAshow up in the processor’s memory space. The Zynq is idealas an educational platform because it gives students a familiarbase on which to start. The ARM Cortex-A9 is capable ofrunning Linux and is supported by the well known Debiandistribution. Additionally, having an FPGA integrated into theSoC provides students with the opportunity to learn how todevelop hardware, and how to interact with it from a softwareperspective.

The ideas from our software framework proved to berelatively simple to apply to actual hardware. Following isa description of how each of the components of the softwareframework corresponds to our hardware oriented framework.

A. Execution Environment

The execution environment is simply the hardware itself. Weare primarily interested in the Linux operating system, whichis well supported on ARM and by Xilinx on the Zynq. Forclassroom use we would provide the students with the initialFPGA image, the bootloader image as well as the Linux Kernelsupplied by Xilinx.

One modification we have make to the hardware platformto support our framework is the addition of an interrupt blockin the FPGA. The FPGA has a number of interrupt signals

that go from the programmable logic to the processor, and wemake use of those to provide a stable timer interrupt to theprocessor. This provides the framework with a way to samplethe state of the operating system at regular, known, intervals.

The timer block in the FPGA simply provides a stableinterrupt to the processor. In order to take advantage of that,and to use it to provide information to the framework, wehave developed a very simple Linux kernel module. Themodule simply registers a handler for the interrupt, whichthen provides the framework with a known location for abreakpoint.

B. Management Interface

The JTAG implementation in the Zynq ARM is supportedby the OpenOCD Project, an open source JTAG compatibleapplication and library. OpenOCD provides our frameworkwith a low-level and powerful interface to the hardwareplatform that we can use to implement our various interfaces.Being open source also means that the software is freelyavailable to students who can both use and learn from it.

To work on real hardware, we need a management inter-face that can support, at a minimum, pausing the executionenvironment in order to extract information. This capabilityis provided by the built-in JTAG support of both the ARMprocessor and the FPGA. Using JTAG, we are able to pause,reset, start and stop the execution environment in a way thatis similar in functionality to the virtualized environments.

We are limited in our ability to create domains. Unlike thevirtual environment where we can create an arbitrary numberof domains with arbitrary configurations, with real hardwarewe are limited to what is connected via JTAG. This leadsto the framework being able to only enumerate existing realhardware domains, not create new ones.

C. System Interface

Running on actual hardware is quite limiting comparedto running in a virtualized environment when consideringwhat information is available. While memory introspectionis available via the JTAG interface, the other methods ofcommunication are not in the same way. We are not able tosend events on hardware interrupts, system calls, or the xchginstructions, which were very useful techniques for tracingsoftware execution in the virtualized environment.

We can, however, mitigate these problems to a limitedextent. Instead of receiving an event on every system call, forexample, we can set a breakpoint on the system calls that weare interested in using OpenOCD. Then we are able to extractinformation from the specific system call or calls that we areinterested in. The same method can be used for hardwareevents and xchg instruction messages by placing breakpointsin locations of interest.

It is worth noting, however, that due to the inconvenienceof identifying all locations of interest and setting breakpointsat them, real hardware is not the best solution for exploratoryinvestigation. In cases where the goal is basic system under-standing it is best to run in a virtual environment where more

information is available. Real hardware is best for cases wherehardware interactions are of interest or there is a limited scopeof investigation. Additionally, real hardware is obviously moreaccurate in its execution, and makes a good verification for thedevelopment work performed in a virtual system.

D. Limitations

Targeting actual hardware allows us to run real applicationson real hardware, however this does have several draw-backswhen compared to running in a virtualized environment. Theprimary limitation is the decreased visibility we have into thesystem that we are monitoring. In a virtualized environmentwe have access to nearly every aspect of the system as wehave complete access to the execution environment itself. Wecan, and did in the case of QEMU, modify the executionenvironment to provide us with information from nearly anypoint in the system. For real hardware we are limited to whatis exposed by the physical hardware itself, in this case overJTAG.

Over JTAG we have the ability to start and stop theprocessor, read and write to arbitrary locations in physical andvirtual memory, and set breakpoints. Additional features wefound useful in the virtualized environment, such as receivingmessages on hardware events or system calls must be buildon top of these. Other features, such as sending messages inresponse to certain instructions are simply not feasible. As aresult, in exchange for the realism of running on real hardware,we trade off the ability to obtain a more complete and nuancedview of the system.

Another significant limitation is the complexity of dealingwith real hardware. Whereas the virtual environment can berun, entirely self contained, under any common operatingsystem, hardware is more limited. First, the hardware itselfis required, the price of which can be a burden for classes.Hardware JTAG support is required as is OpenOCD, whichcan be non-trivial to set up on non-Linux operating systems.This can be problematic as the goal is to help students learn,not to place more barriers in their way. As a result, much moresupport for the teachers and assistants is typically required.

One final limitation we encountered is that of speed. Read-ing and writing to memory from the virtualized environment isnearly imperceptibly fast while we found that reading a singlevalue from our hardware platform took nearly 500ms in maycases. This is likely a result of a number of things particularto our hardware platform. The JTAG interface on the boardthat we used, the Zybo, is not a direct JTAG connection.The JTAG interface is first converted to a serial interfacewhich is then connected through USB-to-serial converted thatis connected to the host. This daisy chained interface addedlatency. OpenOCD then communicates via the serial interfaceand opens up a socket connection that applications can makeuse of. A dedicated, hardware, JTAG connection would likelyalleviate many of the speed issues we encountered and will bethe subject of future exploration, though Andrus et.al. foundsimilar speed limitations due to hardware debugging whenteaching an operating systems class using Android [9].

Fig. 3. Experimental hardware setup.

VI. VISUALIZATION

A hardware example taken from a planned laboratory as-signment can be seen in figure 3. In this assignment, studentsare asked to implement a driver interface to an LED panel thatis mapped into the processor memory. The kernel interface isa simple x, y, limited to a 5x5 grid, coordinate pair and a(r, g, b) color triplet.

We wanted to provide students with a way to see whatis happening with the system in real-time by displaying thevalues from both the kernel call as well as the resultingmemory values. To present this information, we start by takinginspiration from FPGA simulation tools, such as Vivado andModelSim. These tools display the simulation results as signalvalues over time and also provide a convenient mechanism tonavigate the results by zooming and panning through time.

While the visualization provided by the simulation tools isuseful for FPGA simulation results, it is not entirely sufficientfor the complex data structures typically found in operatingsystems and embedded applications. Many structures cannotbe properly represented just by their values; they simplycontain too much information to display in an understandableway.

Instead, we provide a simplified view in the signal displayand a graphical representation in a separate portion of thedisplay. When the user clicks on a signal at a given mo-ment in time, the graphical representation will be updatedto reflect the item that was clicked on. This does have thedisadvantage that any new data structure would need a newgraphical representation, which may be beyond the capabilitiesof students. However, when incorporated into new softwarelabs and exercises this is not a serious issue and can beprepared for ahead of time.

A simple GUI demonstrating this idea for the LED gridexample can be seen in figure 4. The values from the kernelcalls, extracted from the running kernel, are shown alongsidethe contents of the memory in the FPGA, extracted from theFPGA blockram. We also show a rendering of what resultshould look like based solely on color values, minus advancedeffects like gamma correction, to the right. We plan to extendthis view to support more information, such as the signalsfor clocking data out to the LED panel. This would help

trouble shoot problems, like the bottom-most values not beingdisplayed in figure 3.

VII. RELATED WORK

Virtualization is a powerful tool and has been applied manytimes and in many ways to computer science education. Kle-ment [10] describes various ways in which virtualization hasbeen applied to classroom use. He considers three basic dimen-sions of virtualization in education. The first is infrastructurevirtualization to consolidate and simplify the administrationof educational infrastructure. The second is teaching aboutthe operation and administration of virtualization technologies.The third dimension is the use of virtualization in educationalactivities themselves.

Along this third dimension, virtualization has been appliedmany times, but typically as a platform to work in, rather thanas a technology to build on. Stewart et.al. discuss developinga virtualization platform to create a network environment onmodest hardware, with the goal being the enabling of moreadvanced networking and administration studied [11]. Niehand Vaill used virtualization as a way to provide all students,even remote ones, with the same platform on which to com-plete assignments using the Linux kernel [12]. Others havefocused on teaching operating systems with specific hardwareconfigurations by running them in a virtualized environmentand measuring things like memory performance, cpu load, andpagefile usage [13], [14].

All of these works involve using virtualization as a wayto run operating systems, either for multiple different hard-ware configurations or to provide isolation for exploration. Incontrast, we are examining using virtualization as a way toenhance tools we provide to students, providing them withbetter insight into the assignments.

While it does not seem common in operating systems educa-tion, virtualization, and specifically virtual machine introspec-tion, has been applied to other areas. Virtualization has beenused extensively in the field of intrusion detection to analyzethe internal state of an operating system without disturbingit [15], [16]. Johnson et.al. use virtual machine introspectionto allow debugging of the entire system stack, from thekernel through user-space servers to high-level web services,simultaneously [17]. We use similar techniques, but with thegoal of obtaining simplifying the information provided andpresenting it in an accessible way.

Finally, we draw a great deal of inspiration from a number ofanalysis tools such as DTrace, systrace and valgrind [18], [19],[20]. These tools require a complete and working userspaceand, in the case of tracing tools, a fairly complex kernelimplementation to support tracepoint injection. Our goal is tocreate versions of these tools that can be used external to therunning system so that an existing user-space is not required.We also seek to remove some of the kernel requirements, suchas code injection points, to alleviate some of the burden ofinitial development.

Fig. 4. Sample visualization GUI.

VIII. CONCLUSIONS

In this paper, we have presented a framework we havedeveloped that is intended to make introductory classes inoperating systems easier and more understandable for students.Our motivation for creating this framework was our desire toprovide students with more advanced tools, such as profilersand tracing frameworks. These tools allow the students tolook into the software and hardware of the systems they aredeveloping to increase their intuition as to how the softwareand hardware work. These tools are typically unavailable foroperating system kernel development as the features theyrequire, such as trace point injection locations or workinguserspace and POSIX system calls, are not available.

We have described our framework, the actions, and types ofcommunication that it supports. We have also described ourinitial implementation of our framework in the context of avirtual execution environment. This is because we find virtualexecution is ideal for software laboratory settings due to thelevel of access virtualization provides to the software executingwithin it, and the hardware that the software is running on.Despite this, virtual execution is not able to fully reproducethe experience of running on real hardware, due to its natureas a software simulation.

To handle hardware aspects, and to provide students withexperience on real hardware, we have created a version ofour framework targeting the Xilinx Zynq. Additionally, wedescribe a sample assignment that makes use of it to displayinformation from both hardware and software in nearly real-time in the same display. Assisting students by providingthem with more advanced analysis and development toolsmakes the process of learning complex topics easier, and low-level software is challenging in that respect. Our framework,developed specifically with our assignments in mind, butapplicable across a wide range of problems, helps to addressthese shortcomings.

REFERENCES

[1] Kara Nance, Matt Bishop, and Brian Hay. Virtual machine introspection:Observation or interference? IEEE Security & Privacy, 6(5), 2008.

[2] Bryan Pane. Simplifying virtual machine introspection using libvmi.Technical report, Sandia National Laboratory, 2012.

[3] Daniel Roggow, Paul Uhing, Phillip Jones, and Joseph Zambreno. Aproject-based embedded systems design course using a reconfigurablesoc platform. In Microelectronics Systems Education (MSE), 2015 IEEEInternational Conference on, pages 9–12. IEEE, 2015.

[4] Fabrice Bellard. Qemu, a fast and portable dynamic translator. Proceed-ings of the USENIX 2005 Annual Technical Conference, 2005.

[5] J. Wang, J. Beu, R. Bheda, T. Conte, Z. Dong, C. Kersey, M. Rasquinha,G. Riley, W. Song, H. Xiao, P. Xu, and S. Yalamanchili. Manifold: Aparallel simulation framework for multicore systems. In IEEE Inter-national Symposium on Performance Analysis of Systems and Software(ISPASS), 2014.

[6] Shivani Raghav, Andrea Marongiu, Christian Pinto, David Atienza, Mar-tino Ruggiero, and Luca Benini. Full system simulation of many-coreheterogeneous socs using gpu and qemu semihosting. In Proceedings ofthe 5th Annual Workshop on General Purpose Processing with GraphicsProcessing Units, GPGPU-5, New York, NY, USA, 2012. ACM.

[7] Daniele Bortolotti, Christian Pinto, Andrea Marongiu, Martino Rug-giero, and Luca Benini. Virtualsoc: A full-system simulation environ-ment for massively parallel heterogeneous system-on-chip. In IPDPSWorkshops, pages 2182–2187, 2013.

[8] Kevin P Lawton. Bochs: A portable pc emulator for unix/x. LinuxJournal, 1996.

[9] Jeremy Andrus and Jason Nieh. Teaching operating systems usingandroid. In Proceedings of the 43rd ACM technical symposium onComputer Science Education, pages 613–618. ACM, 2012.

[10] Milan Klement. Models of integration of virtualization in education:Virtualization technology and possibilities of its use in education.Computers & Education, 105:31–43, 2017.

[11] Kyle E Stewart, Jeffrey W Humphries, and Todd R Andel. Developing avirtualization platform for courses in networking, systems administrationand cyber security education. In Proceedings of the 2009 SpringSimulation Multiconference, page 65. Society for Computer SimulationInternational, 2009.

[12] Jason Nieh and Chris Vaill. Experiences teaching operating systemsusing virtual platforms and linux. ACM SIGOPS Operating SystemsReview, 40(2):100–104, 2006.

[13] Dalibor Dobrilovic and Zeljko Stojanov. Using virtualization softwarein operating systems course. In Information Technology: Research andEducation, 2006. ITRE’06. International Conference on, pages 222–226.IEEE, 2006.

[14] Fan Yile. Utilizing the virtualization technology in computer operatingsystem teaching. In Measuring Technology and Mechatronics Automa-tion (ICMTMA), 2016 Eighth International Conference on, pages 885–888. IEEE, 2016.

[15] T. Garinkel and M. Rosenblum. A virtual machine introspectionbased architecture for intrusion detection. Proceedings of Network andDistributed Systems Security Symposium, 2003.

[16] Xuxian Jiang, Xinyuan Wang, and Dongyan Xu. Stealthy malwaredetection through vmm-based ”out-of-the-box” semantic view recon-struction. In Proceedings of the 14th ACM Conference on Computerand Communications Security, 2007.

[17] David Johnson, Mike Hibler, and Eric Eric. Composable multi-leveldebugging with stackdb. In Proceedings of the 10th ACM SIG-PLAN/SIGOPS international conference on Virtual execution environ-ments, pages 213–226. ACM, 2014.

[18] Brendan Gregg and Jim Mauro. DTrace: Dynamic Tracing in OracleSolaris, Mac OS X, and FreeBSD. Prentice Hall Professional, 2011.

[19] Niels Provos. Improving host security with system call policies. InUsenix Security, volume 3, page 19, 2003.

[20] Nicholas Nethercote and Julian Seward. Valgrind: a framework forheavyweight dynamic binary instrumentation. In ACM Sigplan Notices,volume 42, pages 89–100. ACM, 2007.