pen-based interactions to support intractive teaching and ...€¦ · web viewitrace final report...

I-TRACE: Final Report APCC – University Politehnica

Pen-based interactions to support interactive teaching and learning of Computer Science topics

ITrace Final Report for ACPP – University “Politehnica” of Bucharest

223434-CP-I-2005-IT-Minerva-M

December 2007

Adina Magda Florea, Eugenia Kalisz, Serban Radu, Irina MocanuUniversity Politehnica of Bucharest

________________________________________________________________________________Grant N. 223434-CP-1-2005-IT-MINERVA-M


Pen-based interactions to support interactive teaching and learning of Computer Science topics

ITrace Final Report for ACPP – University “Politehnica” of Bucharest

Adina Magda Florea, Eugenia Kalisz, Serban Radu, Irina MocanuUniversity Politehnica of Bucharest

[email protected], [email protected], [email protected], [email protected]

Executive summaryThe recent availability of pen-enabled devices such as Tablet PCs, pen-based (mobile)

devices, or interactive whiteboards fosters the opportunity of a new generation of natural interfaces between the user and the computing device. The new types of interfaces supported by pen interaction allows novel approaches to human usage of computers, allowing the development of both novel methods of problem solving and enhanced applications in various areas., including education. The pen enabled devices have the potential to significantly change the educational process by providing new dimensions of classroom interactions based on digital ink and drawing tools for writing, sketching, and drawing, and for real-time collaboration.

The pen technology can offer significant improvements in computerized learning environments through the development of systems that support participative and collaborative learning. Such systems encourage an active type of learning, the interaction between students and instructors, new possibilities for electronic assignments, and better motivate students in their learning endeavor. Therefore, it becomes important to study how digital ink can be used in the learning process, how it can support different learning and teaching styles, and which pedagogical approaches can benefit from the use of pen-based techniques. When integrating digital pen technologies, educators must re-think their teaching approach, must understand how to produce and best take advantage of new teaching resources and must be able to develop and follow new pedagogical approaches.

The ITrace project gave us the opportunity to study and try to give some answers to the above mentioned challenges by gaining experience in using pen-based interactions in several learning contexts while teaching computer science topics, developing enhanced learning scenarios and studying the impact of graphical interaction on students’ learning performances, learning evaluation, and learning styles.

Specifically, we have focused our work on several issues, namely: initial studies and surveys on how computer science teaching and learning can benefit of pen interactions, on one side, and on available software to support these interactions, on the other side; study of the impact of hand-written note-taking, sketching, and graphical annotation on learner's preferences, learning styles, and the provided added value; use of pen based input and graphical interaction for creating cognitive maps; development of an interactive course and assessment module on Data Structures and Algorithm; development of an interactive course


mailto:[email protected]




and assessment module on selected topics on Artificial Intelligence; integration of learning materials in a Learning Management System; study of the impact of graphical interaction on students’ learning and assessment; development of an interactive environment allowing pen annotations and sketching, and the construction of concept maps; development of relevant good practices of how to use pen-based interaction to increase the effectiveness of learning.

As a conclusion of our work, we think that pen based interactions and digital ink technology have the potential to significantly change pedagogical approaches, production and delivery of learning content, and the quality of student teacher interaction. We also think that our pedagogical experiences and developed learning resources are contributing to the current knowledge of using this new technology in education.

During our work, we found that the overall response to our pedagogical experiences was positive and that the students enjoyed being part of the experiments, obtained improved learning performances, were more motivated during the class and even, some of them, were planning to buy digital ink enhanced devices of their own.

However, we share the view that there are some concerns related to: the deployment of the technology, in particular connected to the availability of associated devices on a large scale; effectiveness of the new pedagogical approach on different aspects of teaching computer science; and commitment of the instructors to efforts required in adequately changing the already available learning resources.

1. Who we are and what we aimed atFounded in 1818, University ″POLITEHNICA″ of Bucharest (UPB, http://www.pub.ro) is

the oldest and most prestigious technical university in Romania. The foremost mission of UPB is to educate students in science and technology by imparting knowledge and practical skills, developing their creative thinking, and preparing them to address the demands of today economy.

At UPB, 24,000 students are studying at Bachelor, Master, and Ph.D. levels in the following fields: electrical engineering, power engineering, automatic control and computer science, electronics, mechanical engineering, system management, aerospace engineering, transports engineering, material science engineering, industrial chemistry, economics engineering, environmental engineering, mechatronics applied sciences.

The University houses 37 Research Centres, among which 4 were recognized as Centres of Excellence at national level and 8 grew into Multi-User Research Infrastructures with the support of the Romania - World Bank Program. UPB has a comprehensive infrastructure with modern research and teaching laboratories and an Intranet/Internet communication network. A Scientific & Technological Park is currently under development at UPB, to bring real-world technology and management issues into its research laboratories and teaching.

The strategic lines of development are in fundamental research: micro and nanotehnology, non-conventional technologies, modeling of biochemistry processes, signal & image processing, intelligent robots, cognitive systems; and in applied research: tribology, environment engineering, clean transport systems, electrical vehicles, composite and


http://www.pub.ro/


“intelligent” materials, wasted water treatment, ionic interchange process, biodegradable polymers.

UPB is firmly integrated in the international academic community and shares the same moral, educational, scientific, and cultural values. Due to its prestige, UPB has bilateral co-operation agreements with 74 universities from Europe, America, Asia and Africa, and is member of international academic organizations such as: CESAER, EUA, IAU, AUPELF-UREF. UPB actively participates in R&D international programmes like: COST, FP5, FP6, CORINT, NATO, etc. Amongst its current priorities, UPB aims at valorizing its human potential and logistic possibilities towards the full integration in the European Research Area.

The Faculty of Automatic Control and Computer Science (www.acs.pub.ro), one of the biggest in UPB, offers undergraduate and graduate programmes in “Computer Science” and “Automatic Control and Applied Informatics.” With over 3400 undergraduate, M.Sc. and Ph.D. students, and 270 faculty members, the Faculty is consistently among the top-ranked in Romania. Recognizing that research is on the critical path to Romania’s integration in the EU, the main goal of the Faculty is to maintain an outstanding record in teaching, research, and innovation in IT and advanced control. Its research excellence has been confirmed at both national and international levels.

Areas of faculty expertise include: high performance computing, distributed systems, VLSI, artificial intelligence, relational databases, graphics, computer networks, human-computer interaction, intelligent control systems, bioengineering, dynamic and real-time systems, industrial process control, signal processing and communication, discrete event systems, diagnosis. The Faculty houses modern laboratories with significant computing resources and advanced technological platforms. Its future research priorities are in intelligent autonomous systems, adaptive enterprises, Grid computing and services, ubiquitous computing and communication.

APCC is the Excellence in Research Center for Automatics, Process Control and Computers and includes several well established research domains and laboratories, both in System Control and in Computer Science.

Founded in 1997, AI-MAS Laboratory (http://turing.cs.pub.ro/ai_mas ) focuses its research on multi-agent systems, with special interest in coordination mechanisms, automated negotiation, multi-agent learning, MAS architectures and autonomy. Members are also involved in researches related to models of affective computing, evolutionary agents, intelligent agents in e-learning, and intelligent agents in CSCW.

The AI-MAS laboratory has been involved in the development of several national and international R&D programmes and grants, and maintains cooperation relationships with similar laboratories and computer science departments in European universities, such as Ecole Polytechnique de l’Université de Nantes, Ecole Nationale Superieure des Mines de Saint-Etienne, Université Paris 13, Free Univerity of Amsterdam.

The AI-MAS Laboratory has been member no. 21 “AgentLink, Network of Excellence for Agent-Based Computing”, EU FP5 and member no. 117 of AgentLink III: EU FP6 Co-ordination Action for Agent Based Computing, EU FP7.


http://turing.cs.pub.ro/ai_mas

http://www.acs.pub.ro/


Members of AI-MAS Laboratory are professors, associate professor, lecturers and assistants. Among the courses they are teaching, we can mention: Programming Languages, Data Structures and Algorithms, Artificial Intelligence, Multi-agent Systems, and others. Some of the AI-MAS members are also teaching course at the Faculty of Engineering in Foreign languages of University “Politehnica” of Bucharest, at the Electrical and Computer Science Department, both in the English Stream and in the French stream.

Several AI-MAS members were involved in the ITrace project and aimed at gaining experience in using pen-based interactions in several learning contexts while teaching computer science topics, developing enhanced learning scenarios and studying the impact of graphical interaction on students’ learning performances, learning evaluation, and learning styles.

In the framework of the ITrace project, we have investigated and try to prove how the new technology of digital ink offers flexibility and a range of pedagogical expressions that can achieve several educational goals, support a more participative attitude of students in learning, and how this attitude impacts the amount and quality of acquired knowledge.

Among our specific goals, we can indicate: the investigation of pen-based input and graphical interaction in teaching and learning to understand how annotational capabilities may be used in educational activities; the impact of using pen based input and graphical interaction for creating cognitive maps; the investigation of the impact of graphical interaction on students’ learning and assessment; the development and deployment of an interactive course and assessment module on Data Structures and Algorithm and its integration into a LMS; the development and deployment of an interactive course and assessment module on Artificial Intelligence; the study of the impact of hand-written note-taking, sketching, and graphical annotation on learner's preferences, learning styles, and the provided added value; and contributions to the development of a reference model on how to use pen-based interaction to increase the effectiveness of learning.

2. Interactive learning: how to learn and teach using the digital penInteractivity in learning is generally recognized as one of the key aspects of improving

students’ performances and motivation during learning. Several papers in existing literature report on the benefits of using pen-based interaction to support interactivity in learning.

2.1 Teaching using the digital pen

The first goal of our work was to investigate the use of pen-based input and graphical interaction to understand how annotational capabilities may be used for educational activities. In order to achieve this goal we have conducted a study of the current experience of using pen based interaction in teaching Computer Science, a study of the existing techniques and pedagogical approaches that try to exploit pen-based interaction to improve learning outcomes and to stimulate a participative learning style, but also a study of the existing software tools that allow pen based interaction. This last study included open source and/or free products and proprietary software products.

Our study was based on several available on-line articles that are presented and summarized at (ITrace: Web: Pen papers, 2007; ITrace: Pen Review).



Slide-based annotations systems are quite wide spread (Anderson et. al., 2004; Simon et. al., 2004). Most of them were born from research projects at different universities cross over the world, and had the goal to integrate new digital ink technologies in systems that are used to deliver lectures. Two relevant examples, in this sense, are:

Classroom Presenter, a system developed at the University of Washington and that can be freely (for educational use) downloaded from the Internet

DyKnow, a project started at the DePauw University and which nowadays is a widespread commercial product, used in active and collaborative learning

These systems, which run on a Tablet PC or a pen-based mobile computer allow the instructor to handwrite over computer-projected slides. The slides and ink are then multicast to other machines for students’ use. Students have the ability to take notes with the pen and to offer real-time anonymous feedback to the instructor. The ink annotated slides can be saved for review after lecture, or made available electronically to the students.

Although digital slide projection is controversial it has a number of advantages, including the ability to structure material in advance, prepare high quality examples and illustrations, easily share and reuse material and facilitates distance learning. These kind of systems seemed to be well received by the students. Their general opinion was that such systems can considerably increase spontaneity in lecture presentation. They also feel that they are encouraged to engage in classroom activities and to work in teams. Exchanging ideas with their colleagues on a particular topic help them to construct new knowledge.

There are no or very few reported experiences on pen-based annotations systems for web pages. There are several examples of textual and graphical annotations systems for web, but none of them had support for digital ink. The best known tools for web annotation are listed at (Web annotation, 2007). It seems that most of them accept textual not graphical annotations.

Two representative examples of such web technologies are: Annotea and OntoMat. These are Semantic Web based projects which use RDF (Resource Description Framework) metadata and OWL(Web Ontology Language)-markups. One big benefit using Semantic Web technologies and metadata is that user generated metadata can be easily combined and reused in many other applications.

The technologies mentioned above keep annotations and bookmarks in specialized objects which can be extended. These objects are web resources that have a URI, contain some RDF metadata and normally include a property referring to some other Web resource. Annotations are used for sharing comments, notes, questions, explanations, discussion threads and so on and provide better collaboration over the web. One main advantage provided by these systems is that all the data can be structured on topics and categories and therefore can be easy manipulated.

The study on existing products dedicated to digital ink annotation included the following products: SATIN – A Toolkit for Informal Ink-based Applications; kAWT; JTablet SketchStudio; OpenOffice SDK; Classroom Presenter; Microsoft OneNote 2003; Corel Grafigo 2; DyKnow Vision; DyKnow Monitor; Adobe Acrobat Standard 7.0; Waba; Ewe; riteForm Local SDK; Groove Workspace. The review of each of the products is available at (ITrace: Pen Software, 2007).


http://annotation.semanticweb.org/tools/ontomat

http://www.w3.org/2001/Annotea


2.2 Pen Annotator

We have developed a software module, called Pen Annotator that takes advantage of pen-based technology to encourage active and collaborative learning. Its main aim is to annotate documents using a digital pen. These annotations can be graphical (underlying lines , circles around a portion of text, bullets, checkmarks or other types of graphical objects), or textual, permitting the user to add hand-written text to the image.

This software is intended to improve natural interaction with electronic documents through the use of pen and digital ink. Its goal is to facilitate annotations on documents allowing handwritten complex notes taken on documents as well as creation of notes from blank pages. Particularly interesting is the “movie mode” feature which allows the user to see how a complex drawing had evolved when it was created.

The following features describe the application: create a blank page where annotations can be added load a locally stored image for annotation annotate the loaded image with different colors and line widths save the annotated image locally and make it available through the web easily switch to next image in the folder or project easily switch back to previous image in the folder or project page erase, stroke erase functionality ask the user if he wants to save a yet unsaved annotation when a switch to another image

is being made allow the user to define a project in which to include as many images as he likes and load

them as a whole just by loading the project allow the user to optionally enter a username used to “sign” his own projects movie mode

- the user has the ability to see the annotations as they progressed when they were made- for this he chooses to enter movie mode when annotating an image, chooses

“automatic save every x seconds” with the ability to set x to any value he wants or “save manually when asked” and then starts annotating the image

- the system will record the progress every x seconds as specified or when the user chooses if manual save is selected

- then, at a later time he can review the image with annotations in steps: first what has been annotated after x seconds, then what the image looked like after another x seconds, etc.

Using the Project menu the user has the ability to define a project in which to include as many slides or images as he likes and load them as a whole just by loading the project. In the right part of the graphical user interface is a Project view window where the user can visualize existing slides/images in the project, can add new images, can remove selected images , can move a previously selected image one position up or one position down in the image hierarchy, all these by clicking the associated buttons.



The application was developed in Java (the user has to have JRE 5.0 installed) and was integrated in a “2in1” application with the concept map builder @Graph presented in Section 3.3.

Figure 1. Use of Pen Annotator

Figure 2. Importing a directory in Pen Annotator

3. Meaningful learning: building concept maps using the penConcept maps offer a method to represent information visually. Concept maps harness the

power of our vision to understand complex information “at-a-glance.” Our aim was to investigate the impact of using concept maps in teaching computer science and how this impact can be influenced by the fact that concepts maps are drawn with a pen instead of using “classical” computer supported drawing tools (mouse, drag and drop figures, etc.)

3.1 Concept maps in learning



Concept maps are represented by diagrams that contain nodes and labeled arrows (links between nodes). The nodes correspond to important concepts in a domain, denoted by one or more words, and are enclosed by a circular or rectangular border but other forms are also possible. The labeled arrows denote a relation between a pair of concepts (nodes) and the label on the link state the relationship between the concepts. The arrow describes the direction of the relationship and reads like a sentence. The combination of two nodes and a labeled arrow is called a proposition. A proposition is the basic unit of meaning in a concept map and the smallest unit that can be used to judge the validity of the relationship drawn between two concepts (Dochy, 1996).

Originally, concept mapping was developed by Joseph D. Novak at Cornell University in the 1980s, as a way to increase meaningful learning in the sciences. Novak based his work on David Ausubel’s theory of meaningful learning that stated that prior knowledge is used as a framework for understanding and learning new knowledge, in other words learning new knowledge is dependent on what is already known (Novak,1991). More specifically, new knowledge gains meaning when it can be substantively related to a framework of existing knowledge rather than being "processed and filed" in isolation according to more or less arbitrary criteria. Concept mapping supports the visualization of such conceptual frameworks and “stimulates prior knowledge by making it explicit and requiring the learner to pay attention to the relationship between concepts” (Jonassen, 1993).

Nowadays concept maps are used in education but also in business, management, and research. Examples of concept maps usage are : note taking and summarizing, knowledge elicitation for individual expert knowledge and team knowledge, knowledge capture, new knowledge creation: e.g., transforming tacit knowledge into an organizational resource, mapping team knowledge, collaborative knowledge modeling and the transfer of expert knowledge, facilitating the creation of shared vision and shared understanding within a team or organization, trainings, increasing meaningful learning, communicating complex ideas and arguments, strategic planning, product development, market analysis, decision making, measurement development, tools to support the interviewing process in knowledge acquisition from experts.

Concept mapping is the strategy employed to develop a concept map. There are many approaches regarding the development of a concept map. Some sustain a hierarchical downward structure with the most enclosing concept at the top of the map and the more specific concepts at the bottom. Other approaches are less restrictive and present a variety of structures, as presented in the category classification of concept maps further on.

Another encountered rule states that general concepts should be enclosed in circles, while the particular instances of objects should be enclosed in rectangles. Another version says that specific examples of objects are not to be represented except for the cases in which there presence is relevant as an example of a given concept that helps clarify its meaning. In this case these are not included in ovals or boxes, since they are specific events or objects and do not represent concepts. As a general rule to embrace most of this approaches, the layout is not as important in constructing a concept map as long as it is hierarchal structured and concepts are represented from general to particular and the relationship between concepts is marked and labeled.



We can identify four categories of concept maps. These are distinguished by their different format for representing information, as follows: Spider concept map. The “spider” concept map is organized by placing the central theme

or unifying factor in the center of the map. Outwardly radiating sub-themes surround the center of the map.

Hierarchy concept map. The hierarchy concept map presents information in a descending order of importance. The most important information is placed on the top. Distinguishing factors determine the placement of the information.

Flowchart concept map. The flowchart concept map organizes information in a linear format.

Systems concept map. The systems concept map organizes information in a format which is similar to a flowchart with the addition of “inputs” and “outputs”.

3.2 Meaningful learning through concept maps

Assuming that knowledge within a content domain is organized around central concepts, to be knowledgeable in the domain implies a highly integrated conceptual structure. Concept maps, then, represent some important aspect of a student’s declarative knowledge in a content domain (Jonassen, Beissner, and Yacci, 1993; White and Gunstone ,1992).

As a learning tool, concept maps can contribute to meaningful learning by knowledge capture, knowledge representation, integration of new knowledge with existing one and even knowledge elicitation (Bareholz and Tamir, 1992; Novak, 1990). Being a form of visual representation, concept mapping has several advantages: visual symbols are quickly recognized, minimum use of text makes it easy to scan for a word, phrase, or the general idea, and visual representation allows for development of a holistic understanding that words alone cannot convey.

The stages to build a concept map can be briefly summarized as follows: Selecting the subject referred to by the concept map. Defining the context for the concept map in the form of a concept or a question

called Focus Question, that specifies the problem or issue the concept map should have to resolve.

Extracting the key concepts in the form of a list. Depending on the concept maps purpose this can be done either from memory or from reviewing the material.

The list of concepts resulted is then ranked from the most general concept to the most specific ones. An intermediary ordered list can be used for this purpose.

After that the concept can be placed on the map respecting the hierarchical order no matter of the chosen layout.

Concepts have to be linked together using meaningful linking words. Cross-references can be added in they exist.

This process results in a primary concept map that is revised afterwards by adding new concepts, links or by moving around existing ones. The final concept map is ready after a number of such revisions.



Intuitively, the use of concept maps to evaluate students’ declarative knowledge structure is appealing. A student’s map construction directly reflects, to some degree, her or his understanding in a domain.

However some barriers prevent their widespread use in teaching and evaluation of students’ knowledge. Some barriers are related to the difficulty of building such a map on a computer by using traditional interfaces, especially if the instructor or the student wants to do this interactively, during the class. Drawing a concept map with a pen helps the person focus on the core of the process, makes the drawing easier and prevents being disrupted by the details of drawing with classical drawing tools.

3.3 The Concept map builder

We have developed a software tool named @Graph that allows the development and management of concept maps and can be run on either classical PCs, for which concept maps are drawn with a mouse, or on Tablet-PC or PCs equipped with a pen, for which concept maps are drawn with the pen. Moreover, the software allows the possibility to combine a concept map designed with pre-defined forms (mouse-based) with pen-based annotation of the map. One of the main advantages of our tools is the possibility to use files in both an internal specific format and in JPEG or GIF formats. This ensures the interoperability of the didactic material with other available software.

When creating a concept map, one can either create a concept map from scratch, or load and enhance an already existing concept map (by adding nodes, links, and/or naming nodes and links), either by using available commands in the upper menu or by using pen-based interaction. When using menu commands, a map can be drawn either from scratch or from a given lists of concepts that can be loaded for the description of a certain key topic.

Figure 3. Menu based built concept map



Figure 4. Pen-based enhanced concept map Figure 5. Free-hand drawn concept map

Among the many functionalities of the software we can note: drawing nodes with different shapes and colors, drawing labeled arrows between nodes, adding keyed text with formatting or free text on nodes and links, selecting, copying and pasting parts of a map or the entire concept map, saving and retrieving concepts maps. Besides these basic functionalities, the software allows also to insert pictures in nodes, expand and collapse nodes, auto-arrange nodes and links in a page, and use template concept maps in designing more complex ones. Figure 3 shows a concept map drawn with menu commands and a map, Figure 4 shows an enhanced concept map with free hand sketching, and Figure 5 shows a concept map drawn entirely in free hand.

4. Pedagogical scenarios: coordinating interactive engagementWe have conceived our experiment to cover two aspects of teaching: lecture teaching and

laboratory work (Florea, Radu, 2007). In both cases, the instructor has a tablet PC connected to a video projector and to the local area network. At the beginning of the class, the students are equipped with tablet PCs and receive the slides of the lecture or the slides corresponding to the laboratory topic, i.e., what assignments they have to do during the laboratory. Usually, the assignments are organized in what we call a project – an ordered collection of slides relevant to the topic.

We have developed scenarios for students’ assignments and assessment in which we included both programming and non-programming exercises. Programming exercises were developed in the normal mode during assessment (i.e., using the associated programming environment), while non-programming exercises implied several activities, among which many were requiring hand-written responses that were input using the digital pen.

Because of the limited number of Tablet PCs that we possessed, we have scheduled that during each lecture, every student has at least three times access to Tablet PCs during the entire lecture. During laboratory assignments and assessments, because of mixing



programming and non-programming exercises, we have scheduled the work in such a way so that all students have access to pen-based interactions.

We have tested our scenarios in two different settings: the first setting was the Data Structures and Algorithm course and the second one was a part of the Artificial Intelligence course. The courses were given at the undergraduate level for the students at the Department of Engineering Science (a department of University “Politehnica” of Bucharest with all teaching given in foreign languages, namely English, French and German), the English stream.

4.1 Pedagogical scenarios

Pedagogical scenarios for lectures

The instructor starts by presenting the learning goals of the class either by using a concept map (previously built) or by using bullet text. Then, she presents the lecture topics by going through the prepared slides and annotating them, when necessary, in different manners:

selection of text by underling or circling group of words, or drawing catch attention symbols in the margin of certain sections of the text;

entering short text as group of words to further explain a concept;

building associations by making links to other items of the presentation, including nodes in the learning goals concept map;

drawing new diagrams or drawing additional elements on diagrams already existing on the slides;

writing or drawing examples or completing partially filled examples on the slides.

During the lecture, the instructor may ask the students to:

develop short exercises in order to practice the acquired knowledge, for example “draw the binary tree obtained from a sequence of keys” or “show how an element will be removed from a linked list by emphasizing pointer modification”;

answer 2-3 short questions to catch misconceptions, for example “when can we remove an element from a stack?”

By the end of the presentation, the instructor may:

draw, interactively, a general concept map of the concepts presented during the class;

mark on that concept map the associated relevant slides in the lecture;

ask the students to draw a concept map for a particular concept taught during the class, discuss and modify with the class one such map;

save the annotated slides together with the topics concept map, to be made available to the students.

According to the teacher preference, the slides may be saved with all drawn annotations (end-project) or gradually, one different slide for every annotation, so as to keep in the slides the sequence in which additions were made (step-by-step project). During the lecture, the



students may take their own notes, including slide annotation. In the end of the lecture, they can save their own annotated version of the slides.

Pedagogical scenarios for laboratory work

Before starting the work, we have organized a preparatory laboratory during which one hour was dedicated to expose the students to the use of pen on the tablet PC, to let them practice with the new device, and to explain to the students different approaches of building concept maps. We have found that it is not always obvious, from the first exposure, how to use the pen and how develop a concept map. Therefore, preparatory exercises and examples greatly enhance the desired result of pen-based interactions and the use of maps as an instructional tool. However, getting acquainted with the pen usage required less time and effort than learning how to properly build a concept map.

During the laboratory, students were asked to perform both programming exercises and exercises that required use of the pen (non-programming exercises), such as:

draw flowcharts or pseudocode of the program they are going to implement;

solve exercises that require depiction of data structures and show each step of its building according to a given algorithm;

illustrate the functioning of an algorithm on a particular instance of input data – step by step following the algorithm;

point out errors in a solution of an assignment acting as a teacher who corrects a paper, namely correct the errors in red and try to explain why the solution is erroneous;

justify a formula presented at the lecture;

draw proof trees;

draw concept maps for topics covered by laboratory work or annotate existing concept maps, for example further develop concepts and relationships;

write the solution of a programming exercise instead of keying and running it.

Pedagogical scenarios for students’ assessment

Student assessment took place either interactively during a limited time frame during laboratory or by giving homework assignments to students and evaluated afterwards by the teacher. To this last aim, access of the students to the laboratory where pen-based enhanced computing devices were available was granted.

Every assignment had an associated number of points and, for the non-programming exercises, corrections were made using digital ink. Assignment feed-back and corrected assignments were given to the students.

The assessment scenario for non-programming exercises mimicked in fact the one that was classically performed with pen and paper for student work or exam papers. The basic advantage of using digital ink was obviously that both students and teacher can keep copies of corrected assignments and review them when necessary. Novelty of having electronic corrections was also an attracting factor.



Pedagogical scenarios for using concept maps

One of the important aspects of our scenario was to have an introductory laboratory (in fact a part of a laboratory) in which to present to the students what a concept map is and how one can develop such a concept map according to one or several structured methods. The stages that one has to follow in order to build a concept map were presented in Section 3.2. We found out that this introduction to concept mapping was essential for the proper exploitation of this pedagogical tool by the students.

We have employed concepts maps in different ways and also the construction of concept maps was different, according to the students’ preferences but also according to what the teacher asked at a particular moment.

The specific contexts in which we employed concept maps were the following:

By the teacher when introducing course goals;

By the teacher when summarizing important concepts;

By the students when asked to present acquired concepts;

By the students in their own learning process (the teacher asked them to use concept maps on their own learning process but this activity was not supervised or evaluated).

The method for creating concept maps by the student varied also. Therefore, we have asked the students or let them choose among several methods, namely:

Draw a concept map from scratch to show a given concept or topic;

Refine and/or complete a concept map from a partially built and/or filled concept map;

Draw a concept map by selecting nodes from a list of important concepts given by the teacher; in this case the students are require to organize and link the concepts presented in the list.

Moreover, the students were free to choose if to draw the concept map using menu commands or freely using pen input.

4.2 Data Structures and Algorithms Course

We have developed and deployed (starting from our previous materials on the subject) an interactive course and assessment module on Data Structures and Algorithms, in which we have applied the scenarios described above for lecture teaching, laboratory work and assessment. The topics covered by the course are presented in what follows. We have made our experiments with a group of 25 people for the first course and with a group of 20 students for the second.

Course content Algorithms complexity Vectors and arrays Linked lists: simple linked lists, double linked lists, operations on lists



Trees: tree traversal, binary trees, binary search trees, operations on trees Hash tables: construction, search

Graphs: directed, undirected, graph traversal

For the laboratory work and assessment exercises, we have conceived two types of exercises:

- programming exercises – not developed using pen (a)

- non-programming exercises (b)

The goals of non-programming work exercises during laboratory/assessment were: Information visualization Data structure recognition Understanding of data structures representation in memory Understanding the concept of an abstract data type regardless of the representation Incremental building of data structures for particular cases Discrimination abilities to highlight relevant parts of data structures Illustration of functioning of presented algorithms on particular cases Development of algorithms in pseudo-code Understanding of algorithms analysis by formula justification Understanding the conceptual relationships among presented concepts Understand conceptual variation in organizing concepts

Figure 6. Examples of DSA exercises



Figure 7. Example of DSA learning goals

Assessment feed-back for non-programming exercises (b) was given using digital ink correction, while assessment feed-back for programming exercises (a) was given in the traditional way.

4.3 Artificial Intelligence Course

We have developed and deployed (starting from our previous materials on the subject) an interactive course and assessment module on Artificial Intelligence, in which we have applied the scenarios described in Section 4.1 for lecture teaching, laboratory work and assessment. The topics covered by the course are presented in what follows. We have made our experimented with a group of 15 people.

Course content (the selected chapters)

Basic search techniques

Informed search techniques

Propositional logic

Predicate logic

Resolution and theorem proving

For the laboratory work and assessment exercises, we have conceived three types of exercises:

(a1) - multiple choice exercises to test understanding of the presented concepts and techniques – not developed using the pen

(a2) - programming exercises – not developed using pen

(b) - non-programming exercises

The goals of non-programming exercises during laboratory/assessment were: Information visualization of the problem search space Understanding of problem representation and solution representation Illustration of functioning of presented algorithms on particular cases Understanding proof tree of Prolog programs Understanding unification of expressions



Understanding theorem proving using refutation resolution Understanding the conceptual relationships among presented concepts Understand conceptual variation in organizing concepts

Figure 9. Examples of AI exercises solutions given by students

Figure 10. Examples from an AI lecture

Assessment feed-back for non-programming exercises (b) was given using digital ink correction, while assessment feed-back for programming exercises (a1, a2) was given in the traditional way.

4.4 Integration in a Learning Management System

We have set up an integration of the DSA course in Moodle (2007). Moodle is a course management system, more precisely a free open source software package designed using sound pedagogical principles, to help educators create effective online learning communities. Moodle has a large and diverse user community with over 330,000 registered users. We use Moodle in the Department of Computer Science as an on-line repository and interaction environment for our taught classes.

Moodle has a large number of features and offers the possibility to develop different instructional management strategies. There are several roles in the system (and one can define new roles), among which the most relevant ones are: administrator, course creator, teacher, non-editing teacher, student, etc.



There are a number of interactive learning activities, among which we can quote: assignments, chat, forum, lesson, quiz, forum, survey, and feedback questionnaires (this last one has to be specifically added as a plug-in). There is also the possibility to add a number of resources, such as files, web pages, text pages, link to files or web pages, etc. ach course homepage generally contains blocks on the left and right with the centre column containing the course content. Blocks may be added, hidden, deleted, and moved up, down and left/right when editing is turned on. Latest news, blogs, upcoming events, and recent activity are a few examples.

Students may be enrolled in a course (or several courses, in fact) and, once enrolled, they have the possibility to access the course resources, post assignments, participate in forums, chats and blogs. The teacher managing a course has the possibility to view students’ posted assignments, grade assignments, post news in forums, answer questions asked by the students, view results of surveys and feed-back questions.

We have defined the following sections for our course:

news forum – place for the teacher and assistants to insert announcements about the course, laboratory and assessments;

course description – description of the course, its aims and its syllabus, including references and grading rules

course feedback – a questionnaire requesting students to send general feedback about the course, how useful it has been, quality if the course, etc.

pen-based enhanced learning feedback – the pen-based enhanced learning questionnaire, second version (see Annex 2) where students were asked to answer the questions regarding the use of pen, their appreciation of the technique, and their learning style;

learning style questionnaire – a link to the Felder and Silverman learning style questionnaire (Section 5.2) where students can take the test in order to find out about their learning style;

questions forum – where students can ask questions about the course, about the assignments, and where both other students and teacher/teaching assistants can respond to these questions, the answer being available to all enrolled students.

We have posted on the LMS the slides of the course, both the initial version of the slides and the annotated version. In time, we have found out, based on student reactions, that only the annotated version was consulted by the students.

The environment permits also the management of student assignments. The teacher or the teaching assistant uploads assignments on the site, with an associated due date. Then students have to submit the answers, in particular they can load a file with the answers. In our case, the students have to submit a file that was previously develop either in a programming environment, for programming exercises, or a file developed using digital ink in which they can give answers to non-programming exercises (Sections 4.2 and 4.3). The environment allows the teacher or the teaching assistant to grade the assignments on-line, to assign grades to assignments (as a number of points from 100), to write general comments about the



assignment and to upload a file with corrections to exercises. Students can then view their grades and associated corrections.

A very useful facility of the LMS was the Feedback questionnaires (Figure 11) that can be defined, which allows the definition of questions containing several types of answers, the marking of questions which have required answers or optional answers, and provides anonymous feedback from the students. The answers are automatically collected and counted by the system, according to the different choices specified in each questions, and the results can be exported as an Excel file.

Figure 11. LMS questionnaire – questions marked in red have required answers

5. Supporting differences: influence of learning styles on teaching and learning

People perceive information differently. Students learn in many ways, either by seeing and hearing, or thinking logically and acting, drawing analogies or reflecting on the overall information that they received while being taught, and so on. Teaching methods also vary as some instructors lecture, others like discuss ideas and approaches, some emphasize memory and others understanding. Ideally, a student’s learning style has to be matched by the teaching style. However, we face two problems: how to find out a student’s learning style and whose responsibility is to best fit a learning style with a learning approach, namely is the teacher responsible for matching different learning styles or is the student responsible to learn according to his/her learning style. Moreover, we should try to investigate if these two problems can be tackled also by use of pen-based input and feed-back.

5.1 Learning styles

A learning style is a student’s consistent way of responding to and using stimuli in the context of learning. Learning styles are “characteristic cognitive, affective, and psychological behaviors that serve as relatively stable indicators of how learners perceive, interact with, and respond to the learning environment” (Keefe, 1979). Learning styles tend to be relatively stable over time, in other words are predictable, but are not static in that there can be some variation from day-to-day, week-to-week, and as one ages (Price, 2004).

Students are characterized by different learning styles, respond well to different types of information and favor different ways of acquiring new knowledge. Teaching methods also



vary (Felder, Brent, 2005). Some instructors lecture, others demonstrate or lead students to self-discovery; some focus on principles and others on applications; some emphasize memory and others understanding. A student learns best when his/her learning style is matched by the teaching style and tend to be frustrated and have poor performances otherwise. To maximize student learning, it is important for instructors to address the variety of learning styles when designing and delivering their lessons.

Although its origins have been traced back much further, research in the area of learning style has been active for around four decades. During that period the intensity of activity has varied, with recent years seeing a particularly marked upturn in the number of researchers working in the area. Also of note is the variety of disciplines from which the research is emerging. Increasingly, research in the area of learning style is being conducted in domains outside psychology--the discipline from which many of the central concepts and theories originate. These domains include medical and health care training, management, industry, vocational training and a vast range of settings and levels in the field of education.

We have investigated several learning style models, among which the VARK model (Flemming and Mills, 1992; Flemming, 2001; Web on VARK, 2006), the Kolb learning styles inventory (Kolb, 1984, Web on Kolb, 2007), the Honey and Mumford Learning Styles (Honey and Mumford, 1986), and the Felder and Silverman learning style model (Felder and Silverman, 1988).

The VARK model identifies four different media and, respectively, four distinct learning styles. These styles are visual, aural, reading/writing, and kinesthetic. The name of the theory is the acronym of these four terms. For people with aural preference speech is the preferred and most efficient way of receiving information. Students with this preference learn best from lectures, discussions etc. People from the reading/writing group prefer to receive information from written or printed words. Students with this preference learn best from textbooks, lecture notes, handouts, etc. Members of the third group, visuals, like information to arrive in the form of graphs, charts, various diagrams etc. They are particularly sensitive to matters like color coding or spatial layout. The last group, kinesthetic, needs concrete, multi-sensory experience. They learn by doing. Students with this preference learn best from practical sessions, field trips, experiments, role playing or simulation, etc. In order to acquire conceptual and abstract material they need it to be accompanied by analogies, metaphors, and real life examples. An immediate addition to this four groups classification are the groups of various multi-mode preferences. People with multi-mode preferences, “lucky ones”, can get information using several ways equally well.

The Kolb learning cycle model of learning suggests that successful learning should pass through the following stages: Concrete Experience (having an experience), Reflective Observation (reviewing the experience), Abstract Conceptualisation (concluding from the experience), and Active Experimentation (planning the next steps). Kolb classifies learners as having a preference for concrete experience or abstract conceptualisation; and for active experimentation or reflective observation. According to Kolb, concrete perceivers absorb information through direct experience - doing, acting, sensing, and feeling; abstract perceivers take in information through analysis, observation, and thinking; active processors make sense of an experience by immediately using the new information; reflective processors make sense of an experience by reflecting on and thinking about it.



The Honey and Mumford learning styles is developed from Kolb’s inventory and learning cycle and has four components: activists; reflectors; pragmatists; and theorists. Activists learn best from activities where they can throw themselves into a task. Reflectors learn best when they can review what has happened. Theorists learn best when they can understand what they have learned as part of a wider picture. Pragmatists learn best when an opportunity presents itself to learn on the job.

The Felder and Silverman learning style model is the one we have chosen and which is presented in the following section.

5.2 Selected approach

We have selected the Felder and Silverman model (Felder and Silverman, 1988; Felder, Felder and Dietz, 2002; Felder and Brent, 2005) for its accuracy and simplicity but also because we have considered it as being appropriate for Computer Science students. The model was originally formulated by Dr. Richard Felder in collaboration with Dr. Linda Silverman, an educational psychologist, for use by college instructors and students in engineering and the sciences, although it has subsequently been applied in a broad range of disciplines.

According to this model, students may be classified along four dimensions:

Active learners understand new information by doing something with it while Reflective learners prefer to think about new information first before acting on it;

Sensing learners like learning facts and solving problems by well established methods while Intuitive learners prefer discovering new relationships and can be innovative in their approach to problem solving;

Visual learners understand new information best by seeing it in the form of pictures, demonstrations, diagrams, etc, while Verbal learners understand new information best through written and spoken words;

Sequential learners understand new information in linear steps where each step follows logically from the previous one, while Global learners tend to learn in large jumps by absorbing material in a random order without necessarily seeing any connections until they have grasped the whole concept.

To asses the student's preferences for one style or another, we have used the Index of Learning Styles Questionnaire, developed by Barbara Soloman and Richard Felder from North Carolina State University.

http://www.engr.ncsu.edu/learningstyles/ilsweb.html

The questionnaire scores students on a scale from 0 to 11 in one direction or another along the four dimensions of learning style mentioned above, according to their answers to a set of 44 questions.

If the score on a scale is 1-3, the learner is fairly well balanced on the two dimensions of that scale.




If the score on a scale is 5-7, the learners have a moderate preference for one dimension of the scale and will learn more easily in a teaching environment which favors that dimension.

If the score on a scale is 9-11, the learners have a very strong preference for one dimension of the scale. You may have real difficulty learning in an environment which does not support that preference.

We have used these measures to cluster the results in 5 areas:

<X>strong = 9-11 <X>med = 5-7 Aver =1-3 on both sides

with <X> being one of the two values along a given dimension

For example, in the chart bellow Sstrong means “Sequential learner” in 9-11, Smed means Sequential in 5-7, Aver means Sequential or Global in 1-3, Gmed means “Global learner” in 5-7, and Gstrong means Global in 9-11 (see Figure 12)

I like using the pen

0

5

10

Sstrong Smed Aver Gmed Gstrong

Sequential to Global learners

Ans

wer

s D

SA1

+ D

SA2

SAADSD

Figure 12. Different responses according to learning style


Learning Styles Results ACT X REF 11 9 7 5 3 1 1 3 5 7 9 11 <-- -->

SEN X INT 11 9 7 5 3 1 1 3 5 7 9 11 <-- -->

VIS X VRB 11 9 7 5 3 1 1 3 5 7 9 11 <-- -->

SEQ X GLO 11 9 7 5 3 1 1 3 5 7 9 11 <-- -->


In order to evaluate the students’ learning styles, we have asked the students to go to the Index of Learning Styles Questionnaire, take the test by themselves, and record their results. Moreover, we have also advised students to re-take the test at the end of our pedagogical experience and try to see if there were any changes in their learning styles (any of the four dimensions). Most of the students took the test but some (rather few did not and were content to auto-evaluate their learning styles along the four dimensions after we have briefly explained the characterizations.

Among the students who took the learning styles questionnaire of Soloman and Felder, about 5% reported that the results obtained did not adequately described how they learn, while the others agreed with the obtained results.

5.3 Learning styles, teaching and learning

We share the view that a professor can not tailor the instructional process to fit every student learning style in a class. However, teaching approaches should be adapted to include elements that cover all learning styles, a particular pedagogical element facilitating learning to those who favor that element according to their learning style and exposing the others to differences in knowledge acquisition.

Active learners are stimulated by programming exercises, as they are required to be attentive to details, get practical experience and focus on experimental thoroughness, which are the hallmarks of such type of learners. Both programming and non-programming exercises are appealing to reflective learners as both types of tasks require creativity, theoretical ability, and some kind of guesswork that characterize reflective learners. To be effective when teaching lectures, the instructor should present a blend of concrete information and abstract concepts, support every abstract idea with associated examples, stimulating thus both type of learners. Sitting through lectures without getting to do anything practical but take notes or listen to the teachers’ explanations is particularly hard for active learners. Interactivity while lecturing, such as short required exercises or student feed-back during the class, helps motivate active learners.

Pen-based interaction is appealing in this context from at least two points of view:

allows a faster response time and the implication of more students in interactivity during classes;

satisfy better the necessity of effectively doing something during a computer science lecture where students have no possibility to effectively develop programs.

Although intuitive learning is a natural human learning process, most teaching styles are deductive (which corresponds to sensing learners), in the sense that the teacher starts from general rules and principles and works down towards examples, by organizing and presenting a material that is already understood. One problem with deductive presentation is that it gives a seriously misleading impression to both sensing and intuitive learners. When students see a perfectly ordered and concise exposition of a relatively complex derivation they tend to think that the author/instructor originally came up with the material in the same neat fashion, which they (the students) could never have done. In order to match both types of learners, a teacher should start with the formulation of at least one problem to be solved, and then present the associated methods. Such an approach play to the intuitive learners strength and it also helps



sensing learners develop facility with their less preferred learning mode. We did not find any aspect along this learning dimension that can be really enhanced by pen-based interaction.

Most teaching styles involve the presentation of material in a logically ordered progression, starting from basic facts and leading gradually the learner towards the big picture. Some students are comfortable with this approach; they learn sequentially, mastering the material more or less as it is presented. Others, however, cannot learn in this manner. Sequential learners follow linear reasoning processes when solving problems; global learners make intuitive leaps and may be unable to explain how they came up with solutions. Sequential learners can work with material when they understand it partially or superficially, while global learners may have great difficulty doing so.

One good strategy to reach global learners is to present the lecture’s goals before presenting the steps, doing as much as possible to establish the context and relevance of the subject matter and to relate it to the students’ experience. Concept maps proves to be an invaluable tool for global learners as they offer the students the possibility to see “the big picture”, to make connections and to highlight what is the most important focus points.

Using digital ink in free hand drawing of concept maps is essential to the acceptance of concept maps both in lecturing and in exercises by both types of learners, sequential and globals. Being able to highlight relevant keywords or giving non-programming exercises which asks to show the functioning of an algorithm help global learners stay more motivated in the class.

Visual learners remember best what they see: pictures, diagrams, flow charts, time lines, films, demonstrations. Verbal learners remember much of what they hear or what they read. They get a lot out of discussion, prefer verbal explanation to visual demonstration, and learn effectively by explaining things to others. As most of the lecture is based on speaking and partially on showing written words, the teacher has to devise ways to stimulate visual learners.

Currently, most teachers make an adequate mixture of diagrams, tables, charts and written words in their slides. However, there is a lack of stimulation of visual learners during laboratories taught in computer science.

From this point of view, we have discovered that non-programming exercises based on pen input had a good effect on simulating visual learners. Drawing concept maps also have a good effect on the learning performance of global learners

6. Performance evaluation: pen-based interactions influence on students’ learning6.1 Approach to evaluation

We have based our evaluation on two aspects of students’ activities and results. One type of evaluation was performed based on questionnaires to which the students answered. Another type of evaluation was based on grades of assessed work done by the students and also on the overall grades obtained by the students.

We have developed a set of questionnaires: one first questionnaire which was used in preliminary evaluations (see Annex 2) and a second one which was used in evaluations and



based on which we have developed our conclusions and interpretations. We shall concentrate here on this second questionnaire. For each question, students have to choose among four possible answers: SA - strong agree, A - agree, D - disagree, SD - strong disagree.

We have divided the questions into several groups, each group being dedicated to a particular targeted aspect of the evaluation. In order to link answers with learning styles we have asked the students to make a self test of their learning styles (Section 5.2) but we also made provisions for the case in which students did not do that.

We have divided the questions in the following sections:

Questions for using digital ink in the class (course + lab) – 6 questions

Questions on slide annotation and pen interaction during lectures – 6 questions

Questions on using concept maps – 7 questions

Questions on pen-based exercises–8 questions

General questions on how students feel about the digital ink – 2 questions

1 question about the student learning style – students have to select answers according to the four dimensions of the Felder and Silverman model. They are suggested to first take the Felder and Silverman test.

For collecting the answers to our pen-enhanced learning questionnaire, we have used, during our several experiments, different ways of feed-back, namely: questionnaire answered on papers (as the ones in Annex 2), on-line questionnaire on the ITrace APCC web site, as presented in Figure 13 (a) and the Feedback facility of the Moodle environment, as presented in Figure 13 (b,c).

(a) Web questionnaire (b) LMS questionnaire

Figure 13. Different ways of presenting the Pen-based enhanced learning questionnaire



(c) LMS questionnaire – on learning styles

Figure 13 (cont). Different ways of presenting the Pen-based enhanced learning questionnaire

We had a number of 25 students, and in the second instance 20 students in the Data Structures and Algorithms course, for which we collected and summarized a number of 45 answers and analyzed the answers according to students preferences and acceptance of the new technology. We had a number of 15 students for the Artificial Intelligence course, for which we collected, summarized and analyzed the answers according to students preferences and acceptance of the new technology. Some conclusions were drawn by comparing responses on students enrolled in different courses, while some other conclusions were drawn based on the collected answers of the 45 plus 15 students.

6.2 Performance evaluation

Performance evaluation according to obtained answers to questionnaires

We have conducted several analyses of pen-based enhanced learning questionnaire results and have grouped the answers according to 4 possible choices: SA, A, D, SD. We have focused on the following aspects that were covered by the different questions in the feedback (by grouping answers to relevant questions to each of the bellow criteria):

(1) General opinion about using the pen during teaching and learning

(2) Slide annotation: teacher vs. student – preference towards slides that are annotated by the teacher vs. slides that are annotated by the students themselves

(3) Interaction in classroom – how much the pen and the associated interactions supported by the pen contributed to increasing level of motivation, attention and student understanding of the presented material

(4) Use of concept maps in class – how much this instructional tool was valued by students when used by the teacher



(5) Use of concept maps in exercises – how much this instructional tool was valued by students when used by themselves in assignments

(6) Programming exercises – how did the students feel about writing programs (or making corrections) with the pen vs. in an on-line development environment

(7) Pen-based exercises – how did the students appreciate the pen-based exercises

(8) Feed-back on corrections – how did the students appreciate feed-back written in free form (or corrections) on their assignment files

We have summarized the results for each group of DSA students and IA students and also summarized the results for the entire poll of students who were involved in the experiments.

Most of the students had a general good opinion about using the pen during teaching and learning, as presented in Figure 14. Although the percentage of Agree choices was lower in the AI course, the combined percentage of Agree and Strongly Agree was higher for the students in the AI course than for those in the DSA course.

Slide annotation was also preferred by a high percentage of questioned students, with a stronger preference towards teacher’s annotations as compared with sudents’ annotations; in fact this preference was higher in case of reflective learners and a bit lower in case of active learners. However, active learners had still a higher preference towards the teacher annotating slides than their own annotation.

Interaction in the classroom was definitely the highest rank category from the point of view of stimulating motivation, attention and retention. Percentages for both disciplines were comparable.

The use of concept maps in the classroom was also favorably appreciated although not as high as the previous criteria, with a definite dominance of appreciation in the case of AI students as compared to the DSA students. The use of concept maps in exercises was less favored by students enrolled in both disciplines. Visual learners, global learners, and intuitive learners appreciated the use of concept maps when presenting the learning material and agreed that concept map help them better understand and retain the presented topics. Sensing learners did not really valued concept maps and found this activity somehow boring. Active learners preferred, in general, drawing a concept map with a pen while reflective learners preferred the use of mouse and predefined forms.

Programs written with pen was strongly rejected, which is understandable, but also corrections to pieces of code was a less favored option, in particular among students in DSA course and regardless of their learning style.

Pen based exercises were generally scored high (about 70% Agree plus Strongly Agree), with a higher percentage for AI students, may be because the emphasis of the discipline is more on mechanisms and techniques of problem solving, which are well supported by non-programming exercises, as compared with the DSA discipline. Most of the students who were in favor of pen based exercises were more oriented towards intuitive and visual learning styles. Reflective learners were also more in favor than active learners, a result which was somehow surprising.



We have somehow expected a positive result to pen-based annotation of assignments; however we were surprised by the high percentage of acceptance of this correction method among students in both disciplines. Still, we consider that the strong appreciation obtained was also because of the novelty of the approach and it is to be seen if, when currently exposed to such an approach, the interest of the students will be still significant.

We are aware that the obtained results and evaluations represent the opinion of small poll of students; results in the long run and taken from larger sets of students may vary. However, we consider that the overall response of the students to the introduction of the new technology and, especially, to the new interactions modalities that were triggered by the technology was definitely positive.


16% 15%27%

68% 65% 60%

12% 15% 13%4% 5% 0%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s SAADSD


84% 80%87%

16% 20%13%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s

SA+AD+SD

Figure 14. General opinion about using the pen during teaching and learning

Performance evaluation according to obtained grades

Another way to evaluate learning performances resulted from our experiment was to compare grades. We have performed two comparisons. The first one was by comparing grades of individual assignments of students during the first week with grades of individual assignments in the last week. Results are presented in Figure 15 (a) for the data Structures and Algorithms and in Figure 15 (b) for Artificial Intelligence. From the figure we can see that bellow threshold grades (failures = between 0 and 50 points) has been significantly reduced, with 0 failures in the DSA case, and the top grades (between 90-100 and 70-90) increased. In the DSA case middle grades (50-7) decreased because of the increase in top grades.

Grades for individual assignments

9%15%

21% 26%

61% 59%

9%

First week Last Week

DSA2

Perc

enta

ge o

f stu

dent

s

90-10070-9050-700-50

Grades for individual assignments

7%13%18% 22%

61% 64%

14%1%

First week Last Week

IA

Perc

enta

ge o

f stu

dent

s

90-10070-9050-700-50

Figure 15. Grade comparison



The second comparison was between the number of students which were placed on different scales of top scores in previous classes with those that were involved in our experiments. The results of this comparison were the following: top 10% increased slightly, between top 10% and above passing threshold we notices a significant increase together with a significant decrease of bellow threshold students. Although the results are highly positive, we do not think that the definite improvement in learning performance is due only to the use of the new technology, although that had a definite contribution. We think that the improved results are also justified by the small number of students on which the teaching team was more focused and spent more time in teaching and coaching activities.

7. Lessons learned: good practices and impact of the new technologyWe have found out that digital ink technology allows teachers to better exploit interactions

during the lectures and to broaden the range and diversity of students’ assignments during laboratory work and assessment.

Learning by doing and active learning can be stimulated during lectures by interactive refinement of presented slides but also by engaging students in short feedback activities, such as short questions to which they can rapidly draw the answer or the drawing of a concept map.

We have already used for a number of years non-programming exercises in our classes, such as the ones presented in Section 4, but obviously the solution was to use paper, as the time necessary to develop such exercises using drawing tools was rather too high. Pen-based techniques allowed us to widen the types of these exercises, to integrate them in an LMS and to significantly decrease to solution time.

The role of these non-programming exercises and of building concept maps in consolidating learning is, according to our opinion, significant, by enhancing skills such as: information visualization, data structure recognition, discrimination abilities, understanding functioning of algorithms, understanding the conceptual relationships among concepts presented in the class, understanding of problem representation and solution representation.

We also found out that digital corrections of assignments have a significant contribution to raising students awareness of their misconceptions and mistakes and also stimulate students to review their corrected assignments more often than in paper correction (in the traditional way). Moreover, such a correcting style allows both the students and the teacher to have access to the corrected material.

One of the main advantages of using digital ink, recognized by both the teachers participating to the experiment and by the students, is definitely the speeding up of a lot of activities. e.g., lecture preparation, non-programming assignments solutions, feedback.

Students and teacher can explore their learning styles and different intelligences. By using on-line sources, each student can determine their individual learning styles and dominant intelligences. In addition, they can gain an insight into how they and others learn. This is very important in understanding why different instructional methods should be employed so that all students can have the greatest opportunity for success. The teacher can then construct a pedagogical approach that can exploit different learning styles but also an activity in which to have students discuss how their learning styles can affect their learning performances.



The timeless question posed by students, “Why do I need to know this stuff?,” needs to be addressed in order to build a positive classroom environment. Students need to be convinced or “sold” that the class will be meaningful to them. For example, in an assessment, the teacher can stress that there is not an absolute answer and the true goal is the development of inquiry skills that can be used throughout life. Having a wide range of exercises, that exploit several student abilities, can contribute to “selling” the taught material.

Common to all types of learners is the ability to effectively communicate. Teachers should be aware of the attributes of effective communication and constantly find ways to include them in their activities. Aspects of effective communication must include different forms of communication, including verbal, textual, and visual. Ideally, the teacher should allow students to select the ways to communicate that suit them best, in order for the students to be able to effectively share their acquired knowledge.

When integrating digital ink technology in courses or laboratory work, computing educators must re-think what they teach students and how they enable students to learn, in such a way as to take best advantage of the new communication resources.

We found out that adopting maps for assessment use needs a common understanding of what a concept map assessment is and whether it provides a reliable and valid measure of a student’s cognitive structure. When used as an assessment tool, concept maps are very appealing but without a precise measure of how “correct” a concept map is they can not be accurately used for grading the students’ knowledge.

When evaluating concepts maps developed by the students we have used both a qualitative measure (how correct the concept map is to the teacher) but also some quantitative measures that we intend to further develop in what can be called a “precise” measure of how “correct” a concept map is. The quantitative measures used were the following: the percentage of “right” node concepts used in a CM drawn by the student, the percentage of “wrong” node concepts used in a CM drawn by the student, the percentage of “meaningful” labeled links among concepts.

8. Conclusions and perspectives8.1 Conclusions

The participation in the ITrace project was a very rewarding one. The project gave us the opportunity both to perform our own evaluations on a new technology and novel approaches, but also to interact with the partners in the project, share ideas, experiences and results.

The aim of APCC-UPB partner was to design a set of pedagogical experiences in order to evaluate the impact of using the pen technology on the effectiveness of the teaching process. Moreover, we aimed at evaluating the impact of this technology on different student learning styles and the role concept maps can play in facilitating student learning depending on his/her learning style. We think that our proposed pedagogical experiences can add to the range of available teaching methods that use this new technology.

During our experiments, we found out that the overall response to our pedagogical experiences was positive and that the students enjoyed both being part of the classes that used digital ink and practicing with the new approaches of input data into the computer. However,



we share the view that there are some concerns related to the deployment of the technology, in particular connected to the availability of associated devices on a large scale (we have more than 1000 students enrolled in Computer Science courses at the Department of Computer Science) and potential downsides at pedagogical level. There are also concerns of usability and concerns from the point of view of the extra work that the instructors have to put in order to rethink their classes for optimal use of pen technology.

One of the connected results of our work was the insight gained in valuating the students’ learning styles and pedagogical practices that are able to accommodate different learning styles. Last but not least, we have found out one extra valuable teaching tool, namely the concepts maps that facilitate meaningful learning and help students in knowledge acquisition and recollection.

8.2 Perspectives

Our participation in the ITrace project open up a lot of perspectives and we do not intend to end our pedagogical experiences of using digital ink once the project is over. We have identified several directions in which we shall continue our activity. One of the most important aspects to sustain pen-based experience in learning is the available technology. As prices are constantly getting down, we assume that Tablet PCs will become quite affordable to get. We are planning to get more funding from local or national funding bodies in order to equip an entire laboratory with 24 tablet PCs, a laboratory that can be used both during laboratory work, and during assignment problem solving by the students.

We are also planning to equip a lecture room with an interactive digital board, which will also facilitate digital ink experiences. However, we consider that a teacher equipped with a Tablet PC and a video projector is in general quite adequate to deliver interactive experiences.

We are also planning to transform the Data Structures and Algorithms course and the Artificial Intelligence course in permanent pen-based enhanced learning experiences in the future. Moreover, we plan to involve other members of our department in our experiments and stimulate the teachers to develop their own course and assignments that includes pen-based facilities. Last but not least, we plan to improve our own developed software in order to allow more interactivity and groupware interaction.

We have also a list of open questions that needs to be further explored and investigated. The basic question, to which this study tried to contribute, is “How pen-based interaction can transform the way we, teachers, teach, and the way students learn?” If in the future (which is in fact the near future) most students will have PCs, with a significant fraction of these being Tablet PCs; therefore we need to take advantage of this technology to create new learning environments, and to exploit the fact that (mobile) Tablet PCs allows a new range of communication modes besides the text based one, both in the classroom and outside of it.

ReferencesAnderson, A. et. al. (2004). Experiences with a Tablet PC Based Lecture Presentation System in Computer Science

Courses. SIGCSE 2004. Retrieved December 2007 from http://www.cs.washington.edu/research/edtech/publications/


http://www.cs.washington.edu/research/edtech/publications/


Bareholz, H. and P. Tamir. (1992). A comprehensive use of concept mapping in design instruction and assessment, Research in Science and Technology Education, 10(1), pp. 37-52.

Dochy, F. J. (1996). Assessment of domain-specific and domaintranscending prior knowledge: Entry assessment and the use of profile analysis. In M. Birenbaum & F. J. R. C. Dochy (Eds.), Alternatives in assessment of achievements, learning process and prior knowledge (pp. 93-129). Boston, MA: Kluwer Academic Publishers.

Felder, R.M. and L.K. Silverman. (1988). Learning and Teaching Styles in Engineering Education, Engr. Education, 78(7), 674-681.

Felder, R.M. and R. Brent. (2005). Understanding Student Differences, Journal of Engineering Education, 94 (1), 2005, pp.57-72.

Felder, R.M., G.N. Felder, and E.J. Dietz. (2002) The Effects of Personality Type on Engineering Student Performance and Attitudes. J. Engr. Education, 91(1), 3-17.

Flemming, N. D. and C. Mills. (1992). Not another inventory, rather a catalyst for reflection. To Improve the Academy, 11:137. Retrieved December 2007 from http://www.ntlf.com/html/lib/suppmat/74fleming.htm

Flemming., N. D. (2001). Teaching and Learning Styles: VARK Strategies, Christchurch, New Zealand, 5th edition, 2001.

Florea, A.M. and S. Radu. (2007). Enhancing Pen-based Experiences with the Use of Concept Maps, First International Workshop on Pen-Based Learning Technologies (PLT 2007), IEEE Computer Society Conference Publishing Service, in print.

Honey, P. and A. Mumford. (1986). A Manual of Learning Styles, Peter Honey, Maidenhead

ITrace APCC Web site. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/ITrace/index.html

ITrace: Concept Maps. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/concept_maps/

ITrace: Pen Review. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/pen_tech/review.html

ITrace: Pen Software. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/pen_tech/software.html

ITrace: Pen Techniques. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/pen_tech

ITrace: Web: Pen papers. (2007). Retrieved December 2007 from http://turing.cs.pub.ro/pen_tech/papers.html

Jonassen, D. H., K. Beissner, and M. Yacci. (1993). Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale, NJ: Lawrence Erlbaum Associates.

Keefe, J.W. (1979). Learning Style: An Overview, in Keefe, J.W., ed., Student Learning Styles: Diagnosing and Prescribing Programs, Reston, Va.: National Association of Secondary School Principals, 1979.

Kolb, D.A. (1984) david a. kolb on experiential learning, Retrieved December 2007 from http://www.infed.org/biblio/b-explrn.htm

Moodle. (2007). Retrieved December 2007 from http://moodle.org/

Novak, J.D. (1990). Concept mapping: A usefull tool for science education, Journal of Research in Science Teaching, 27(10), pp. 937-949.

Novak, J.D. (1991). Clarify with concept maps: Atool for students and teachers alike. The Science Teacher, 58 (7), 45-49.

Price, L. (2004). Individual differences in learning: Cognitive control, cognitive style, and learning style. Educational Psychology, 24 (5), 681-698.

Ruiz-Primo, M. A., S.E. Schultz, and R. J. Shavelson. (1997). Concept Map-Based Assessment in Science: Two Exploratory Studies, CSE Technical Report 436, CRESST/Stanford University.

Simon, B. et. al. (2004). Preliminary Experiences with a Tablet PC Based System to Support Active Learning in Computer Science Courses, ITICSE’04, June 28–30.


http://moodle.org/

http://www.infed.org/biblio/b-explrn.html

http://turing.cs.pub.ro/pen_tech/papers.html

http://turing.cs.pub.ro/pen_tech

http://turing.cs.pub.ro/pen_tech/software.html

http://turing.cs.pub.ro/pen_tech/review.html

http://turing.cs.pub.ro/concept_maps/

http://turing.cs.pub.ro/ITrace/index.html

http://www.ntlf.com/html/lib/suppmat/74fleming.html


Soloman, B. and R. Felder. (2007). Index of Learning Styles Questionnaire, Retrieved December 2007 from http://www.engr.ncsu.edu/learningstyles/ilsweb.html

Web annotation. (2007). Retrieved December 2007 from http://annotation.semanticweb.org/annotationtool_view

Web on Kolb (2007). The experimental learning cycle, Retrieved December 2007 from http://www.learningandteaching.info/learning/experience.html

Web on VARK. (2006). VARK: A guide to learning styles, Retrieved December 2007 from http://www.vark-learn.com/english/index.asp

White, R. T. and R. Gunstone. (1992). Probing understanding. New York: Falmer Press.


http://www.vark-learn.com/english/index.asp

http://www.vark-learn.com/english/index.asp

http://www.learningandteaching.info/learning/experience.html

http://annotation.semanticweb.org/annotationtool_view



Annex 1: Charts


16% 15%27%

68% 65% 60%

12% 15% 13%4% 5% 0%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s SAADSD


84% 80%87%

16% 20%13%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s

SA+AD+SD

Interaction in the classroom

25% 27%33%

68% 65% 62%

6% 7% 5%1% 1% 0%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s SAADSD

Interaction in the classroom93% 92% 95%

7% 8% 5%

DSA 1 DSA 2 IA

CS Topics

Ans

wer

s

SA+AD+SD

Slides annotation

21%15%

59%51%

19%30%

1% 4%

Teacher Student

Actor

Ans

wer

s SAADSD

Slides annotation

80%

66%

20%

34%

Teacher Student

Actor

Ans

wer

s

SA+AD+SD



Annex 2: QuestionnairesA2.1 Feed-back questionnaire No.1 – quality of pen interaction (revised form)

“Pen-based enhanced learning” PollThe following set of questions are aimed to evaluate the impact of using the digital ink on enhancing the pedagogical experience, of using the pen on drawing concept maps, and of using concept maps in teaching students characterized by different learning styles.There are 30 questions. In questions number 1 to 29 you have to choose one answer among four possible ones: SA - strong agree, A - agree, D - disagree, SD - strong disagree.In question 30 you are asked to select the learning style that describes you best. In order to be answer this question you may want to first take the Learning Style Test at http://www.engr.ncsu.edu/learningstyles/ilsweb.html Alternately, or you may read the materials on learning style and figure out by yourself which learning style describes you best. Please be aware that question 30 will require you to input a score. The highest the score, the more dominant the feature is.In the end you are free to input any of your comments regarding your pen-based learning experience.Good luck!

Some information about you (your name is not asked!)

Age: 15-20 21-25 26-30

Sex: F M

Background:

Student in Computer Science

Student in an Engineering Discipline

Graduate in Computer Science

Graduate in an Engineering Discipline

Have you any previous experience in using digital ink Yes No

Mother tongue: Romanian English French

Study language: Romanian English French

Now the questionsQuestions for using digital ink in the class (course + lab)

1. Using digital ink in the class is enjoyable SA A D SD




2. Using digital ink in the class is stressful SA A D SD

3. I like using the pen during the class SA A D SD

4. I am more attentive during the class when I use the pen SA A D SD

5. Using the pen distracts me SA A D SD

6. I am more motivated to learn if learning involves pen interaction SA A D SD

Questions on slide annotation and pen interaction during lectures

7. Having the possibility to annotate the slides helps me retain knowledge SA A D SD

8. I do not like to annotate the slides SA A D SD

9. I like when the teacher makes annotation on slides during the class SA A D SD

10. I learn better if I can review teacher's annotated slides SA A D SD

11. I learn better if I can review my own annotated slides SA A D SD

12. Short pen-based exercises asked during the lecture helps me better understand the presented topics SA A D SD

Questions on using concept maps

13. I prefer having the lecture goals presented as a concept map SA A D SD

14. I prefer having the lecture goals written interactively at the beginning of the class SA A D SD

15. The concept map helps my understanding of the topic



SA A D SD16. I prefer to build the concept map by selecting concepts from a given list SA A D SD

17. I prefer building a concept map from scratch SA A D SD

18. I prefer drawing the concept map with a pen rather than with a mouse SA A D SD

19. Building concept maps is boring SA A D SD

Questions on pen-based exercises

20. I prefer exercises which involves pen-based input SA A D SD

21. I prefer programming exercises using key input SA A D SD

22. I consider that pen-based exercises help me better understand the topic SA A D SD

23. I prefer a combination of pen-based exercises and programming exercises SA A D SD

24. I prefer writing a pseudocode with a pen rather than using key input SA A D SD

25. I prefer making corrections to a piece of code with a pen rather than using key input SA A D SD

26. Digital correction of my exercises is clearer than text based or on paper SA A D SD

27. I review more frequently my corrected exercises if they are in digital form (pen-based correction and pointing of errors) SA A D SD

General questions on how you feel about the digital ink

28. I like using the pen SA A D SD

29. If I buy my own laptop I prefer it to be a tablet PC or to have a form of graphical interaction



SA A D SDQuestion about your learning style

30. My learning style is (select one answer in each group):

Active vs. Reflective: Active learners understand new information by doing something with it while Reflective learners prefer to think about new information first before acting on it.

Active learner score 9-11 score 5-7 score 1-3Reflective learner score 9-11 score 5-7 score 1-3I do not know

Deductive vs. Intuitive: Deductive learners like learning facts and solving problems by well established methods while Intuitive learners prefer discovering new relationships and can be innovative in their approach to problem solving.

Deductive learner score 9-11 score 5-7 score 1-3Intuitive learner score 9-11 score 5-7 score 1-3I do not know

Visual vs. Verbal: Visual learners understand new information best by seeing it in the form of pictures, demonstrations, diagrams, etc, while Verbal learners understand new information best through written and spoken words.

Visual learner score 9-11 score 5-7 score 1-3Verbal learner score 9-11 score 5-7 score 1-3I do not know

Sequential vs. Global: Sequential learners understand new information in linear steps where each step follows logically from the previous one, while Global learners tend to learn in large jumps by absorbing material in a random order without necessarily seeing any connections until they have grasped the whole concept.

Sequential learner score 9-11 score 5-7 score 1-3Global learner score 9-11 score 5-7 score 1-3I do not know

Note: On learning style is not better than another. It just explain the way you learn best

Input your free comments………………………………………………………………………………………………

………………………………………………………………………………………………



………………………………………………………………………………………………

………………………………………………………………………………………………

………………………………………………………………………………………………

Thank you for taking this test

A2.2 Feed-back questionnaire No.1 – quality of pen interaction (initial form)

Pen-based enhanced learning PollThe following set of questions are aimed to evaluate the impact of using the digital ink on enhancing the pedagogical experience, of using the pen on drawing concept maps, and of using concept maps in teaching students characterized by different learning styles.There are 25 questions. In questions number 1 to 23 you have to choose one answer among four possible ones: SA - strong agree, A - agree, D - disagree, SD - strong disagree. In question 24 you are asked to select the learning style that describes you best. In order to be able to answer this question you may want to first take the Learning Style Test at http://www.engr.ncsu.edu/learningstyles/ilsweb.html or you may read the available materials on the site about learning style and figure out by yourself which learning style describes you best.In the last question you are able to input any free comments regarding your pen-based learning experience.Good luck!

Some information about you (your name is not asked!)

Age: 15-20 21-25 26-30

Sex: F M

Background:

Student in Computer Science

Student in an Engineering Discipline

Graduate in Computer Science

Graduate in an Engineering Discipline

Have you any previous experience in using digital ink Yes No

Mother tongue: Romanian English French




Study language: Romanian English French

Now the questions1. Using digital ink when presenting the class is enjoyable SA A D SD

2. Using digital ink when presenting the class is stressful SA A D SD

3. I prefer having the lecture goals presented as a concept map SA A D SD

4. I prefer having the lecture goals written interactively at the beginning of the class SA A D SD

5. The concept map helps my understanding of the topic SA A D SD

6. I prefer to build the concept map by selecting concepts from a given list SA A D SD

7. I prefer building a concept map from the presented material SA A D SD

8. I prefer starting from a given basic concept map and add concepts and links SA A D SD

9. I prefer drawing the concept map with a pen rather than with a mouse SA A D SD

10. Building concept maps is boring SA A D SD

11. I like using the pen during the class SA A D SD

12. I am more attentive during the class when I use the pen SA A D SD

13. Using the pen distracts me SA A D SD

14. Having the possibility to annotate the slides helps me retain knowledge SA A D SD

15. I do not like to annotate the slides



SA A D SD

16. I like when the teacher makes annotation on slides during the class SA A D SD

17. I learn better if I can review teacher's annotated slides SA A D SD

18. I learn better if I can review my own annotated slides SA A D SD

19. I prefer exercises which involves pen-based input SA A D SD

20. I prefer programming exercises using key input SA A D SD

21. I prefer writing a pseudocode with a pen rather than using key input SA A D SD

22. I prefer making corrections to a piece of code with a pen rather than using key input SA A D SD

23. If I buy my own laptop I prefer it to be a tablet PC SA A D SD

24. I am more an (select one answer in each row):

Active learner Reflective learner I do not know

(Active learners understand new information by doing something with it while Reflective learners prefer to think about new information first before acting on it.)

Deductive learner Intuitive learner I do not know

(Deductive learners like learning facts and solving problems by well established methods while Intuitive learners prefer discovering new relationships and can be innovative in their approach to problem solving.)

Visual learner Verbal learner I do not know

(Visual learners understand new information best by seeing it in the form of pictures, demonstrations, diagrams, etc, while Verbal learners understand new information best through written and spoken words.)

Sequential learner Global learner I do not know



(Sequential learners understand new information in linear steps where each step follows logically from the previous one, while Global learners tend to learn in large jumps by absorbing material in a random order without necessarily seeing any connections until they have grasped the whole concept.)

25. Input your free comments………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………………

Thank you for taking this test


pen-based interactions to support intractive teaching and ...€¦ · web viewitrace final report...

Documents