supervision typology in computer science engineering capstone projects

Journal of Engineering EducationOctober 2012, Vol. 101, No. 4, pp. 679–697

© 2012 ASEE. http://www.jee.org

Supervision Typology in Computer Science Engineering Capstone Projects

CÉSAR DOMÍNGUEZ PÉREZ, ARTURO JAIME ELIZONDO, FRANCISCO J. GARCÍA-IZQUIERDO, AND JUAN JOSÉ OLARTE LARREA

University of La Rioja, Spain

BACKGROUND Undergraduate students in computer science engineering undertake a capstone project that should integrate the specific knowledge and skills acquired during their studies. The advisor assigned to supervise the process undertakes a multifaceted commitment that varies among advisors.

PURPOSE This study develops and validates an instrument, and then utilizes it to determine the different styles of supervision in computer science engineering capstone projects.

DESIGN/METHOD A questionnaire was developed to survey capstone project advisors at a university during the past two years. A total of 109 surveys were successfully collected. A combination of multivariate statistical methods, such as factorial and cluster analysis, was employed.

RESULTS This study distinguished seven main supervision factors: technology, arrangements, keep alive, execu-tion, meetings, management, and reports. Then, six supervision styles were identified according to the advisor’s varying degree of involvement in each factor: student alone, execution focused, global supervi-sion, management focused, technological mentoring, and process focused. To further characterize these styles, we compared their applications focusing on the type of student, the grade obtained, the project duration, and the amount of time devoted by the advisor.

CONCLUSION We have determined the main factors in capstone project supervision and characterized different styles of supervision according to these factors. These supervision styles can help advisors recognize ways to proceed in the supervision of capstone projects.

KEYWORDS

Capstone project, computer science engineering, supervision typology

INTRODUCTION

Currently, computer science engineering degree programs often expect an undergraduate student to undertake a capstone project (ACM/IEEE-CS, 2001; ACM/IEEE-CS, 2008; Dutson, Todd, Magleby, & Sorensen, 1997; Clear, Goldweber, Young, Leidig, & Scott, 2001). This experience helps the student tackle real future problems with a greater likeli-hood of success. The ACM/IEEE-CS recommends that project development be included as part of the computer science engineering curriculum (CS490, ACM/IEEE-CS, 2001). This project should integrate the specific knowledge and skills acquired over the course of

680 Domínguez Pérez, Jaime Elizondo, García-Izquierdo, & Olarte Larrea

a student’s studies, along with other orthogonal aptitudes required in professional work. The latter include searching for and synthesizing information, making oral presentations, writing reports, and creating prototypes. The student must make diverse decisions regard-ing the technical, methodological, planning, and management dimensions of the project. The experience should be related to the future professional career as much as possible. Both the software industry and students themselves assign great importance to capstone projects and consider them excellent “calling cards” that facilitate a recent graduate’s inte-gration into the professional field ( Joy, 2009; James, Hawick, & James, 2005).

Different academic systems may implement capstone projects in a variety of ways. It is usually conceived as a year- or half-year-long experience. The project can be conducted individually or as a team, and designed as an academic project or done within a corporate setting ( James et al., 2005; Dutson et al., 1997; Clear et al., 2001). Essentially, it must be an engineering project in which the student designs a solution to a problem (ACM/IEEE-CS, 2008; Clear et al., 2001).

In the University of La Rioja, Spain, the capstone project is treated as a regular course without lectures, reserved for the last year of the degree program. In general, the project workload is about 600 hours, not necessarily restricted to one academic year. The cap-stone project coexists with other courses in the curriculum for the same year. Students usually prefer to pass all their other courses before actually beginning their projects. Gen-erally, the project work includes the analysis, design, implementation, and testing of an information-based problem. With the assigned advisor, each student develops the work individually and freely designs the topic and methodology to be followed. A university committee reviews all project proposals in order to ensure their suitability and that they have an adequate and similar level of complexity. Although the scope can vary, a signifi-cant proportion of projects follow comparable guidelines. When the project is complet-ed, the student presents a portfolio compiling the written deliverables and the product itself (when appropriate). Additionally, the university organizes an oral defense, in which the student presents the more relevant contents of the work to a three-person committee. Some established evaluation criteria are used, where the technical solution represents about 40% of the grade, the report quality about 30%, management about 20%, and the oral defense about 10%.

An advisor is assigned to each student project, but the advisor’s duties are not clearly defined. Project supervision involves a variety of time-consuming tasks, including tech-nical assistance, meetings, and reviewing documentation and other deliverables. The ad-visor’s responsibility entails concurrent roles as a monitor, mentor, sponsor, manager, and confidant (Clear et al., 2001; Fincher, Petre, & Clark, 2001; Joy, 2009). Success in super-vision depends largely on finding the suitable level of involvement. Students should nei-ther feel alone nor over-assisted ( James et al., 2005). Supervision also differs according to where the advisor usually provides support, for example, in initial planning or technolog-ical matters.

To our knowledge, no supervision typology for engineering capstone projects exists in the relevant literature. The main objectives of this study are to develop and validate an in-strument to determine a supervision typology of computer science engineering capstone projects; determine the main dimensions present in the capstone project supervision ac-tivity; ascertain the main supervision styles characterized by different levels of involve-ment in the previously mentioned dimensions; and study the relationship between stu-dent project results and supervision styles, with the objective of supplementing the styles’ characterization.

Supervision Typology in Capstone Projects 681

RELATED WORK

In our review of the literature, we found few studies devoted to undergraduate capstone project supervision. For this reason, and to find some ideas transferable to our field of study, we explored the literature on supervision in other academic activities in which a similar advisor/supervised-student schema is followed.

Undergraduate capstone projects and Ph.D. dissertations share common tasks, such as review and evaluation of literature, writing structured and coherent reports, justifica-tion of methods and processes employed, and creative thinking (Bouki, 2007; James et al., 2005). However, there are obvious differences, such as the work load and quality, stu-dent maturity, and, above all, the necessary new contributions expected of a research project. The differences between research and undergraduate capstone supervision sug-gest that each one merits methods and styles adapted to their objectives and peculiarities (Todd, Smith, & Bannister, 2006). Notably, there seems to be much less literature on un-dergraduate supervision than on doctoral supervision (Todd et al., 2006; Greenbank, Penketh, Schofield, & Turjansky, 2008).

Research supervision, a related field widely studied, is a multi-factorial process ac-cording to Delany (2008). An effective supervisor must be a capable researcher exem-plifying good practices that help students gain research skills (Donnelly & Fitzmau-rice, 2009; Fraser & Mathews, 1999). A variety of models have been proposed for effective research supervision (Delany, 2008; Donnelly et al., 2009; Lee, 2010). Sets of supervision styles have also been identified. For instance, Gatfield (2005) considers the support and structure level and distinguishes between laissez-faire and contractual styles. Regarding personal interaction, he also recognizes the pastoral style and its op-posite the directorial but argues that neither is right or wrong. Lee (2010) identifies five research supervision paradigms. The functional emphasizes planning, acquiring resources, task completion, and monitoring. The enculturation paradigm aims for the student to become a member of a discipline, whereas critical thinking encourages the student to look for propositions and arguments for and against them. The emancipa-tion paradigm is a personal, transformative process, and the relationships paradigm concerns emotional intelligence. We have not found any typology for capstone project supervision and its possible influence on students’ success in capstone projects. Our work is related to the first of Lee’s paradigms and studies the functional aspects of capstone project supervision.

Most research on capstone project supervision concurs that the advisor-student relationship is a key factor for success ( James et al., 2005; Clear et al., 2001; Clark & Boyle, 1999). A variety of practical advice has been offered to supervisors according to the type of capstone project, such as individual or group, academic or on behalf of a corporation (Bouki, 2007; Malik, Khusainov, Zhou, & Adamos, 2009; Ho, 2003; Scott, 2008; Farr, Lee, Metro, & Sutton, 2001). Advice has been offered for particular aspects of project development, including documentation (Keogh and Venables, 2009) and communication (Malik et al., 2009). A best practices repository has also been created (Clear et al., 2001). Some studies emphasize personal aspects ( James et al., 2005; Marin, Armstrong, & Kays, 1999), whereas others enumerate various advisor roles (Clear et al., 2001; Fincher et al., 2001; Joy, 2009; Scott, 2008; James et al., 2005).


RESEARCH DESIGN

With our objective of creating a typology of supervision, we designed the questionnaire shown in Figure 1 (also available online at http://surveymonkey.com/s/YCFD2C6). The initial pool of items for the survey’s first section, Advisor Involvement, was determined by reviewing the literature on project management, software engineering development, and supervision of undergraduate projects and dissertations (Project Management Institute, 2008; Pressman, 2010; ACM/IEEE-CS, 2001; ACM/IEEE-CS, 2008; Fincher et al.,

FIGURE 1. Capstone projects supervision questionnaire.


2001; James et al., 2005; Bouky, 2007; Joy, 2009; Todd et al., 2006; Armstrong, 2004; Clear et al., 2001; Ho, 2003; Marin et al., 1999; Dutson et al., 1997). As a result, the ques-tionnaire items covered a wide range of functional aspects concerning the supervision of computer science engineering capstone projects. The wording of each item was designed to ascertain the advisor’s involvement in that particular item. Involvement is understood as the advisor’s intensity in engaging in the tasks of guiding and supervising. The intensity for each task can vary from leaving the student alone to diligent labor by the advisor. For example, we match the lower-effort level with situations where the advisor completely delegates the development of certain tasks, decision making, or monitoring deadlines to the student. On the other hand, the higher-effort level corresponds to situations where the advisor closely supervises task performance, greatly influences decision making, or strictly monitors the timetable.

The objective was to design an instrument with high content validity that exhausted the domain without redundancy. Initially, we produced a pool of 28 items. To avoid omit-ting important aspects, we asked several university professors with experience in capstone supervision to review the items. They recommended making the wording more precise to avoid misinterpretation, eliminating two items of insignificant importance, and adding a new item. As a result, we produced a final version of 27 four-point Likert-type items, la-beled from “strongly disagree” to “strongly agree.” There is no middle option, creating a forced-choice method. This section has a global measure item (no. 27) to analyze the con-current validity of the instrument.

The Advisor Spent Time section asks how much time the advisor devoted to aspects such as meetings, communication with the student, revision of reports, and overall time commitment. The Student’s Skills section deals with features such as autonomy, manage-ment, technology and methodology, meetings and communication, writing, and public presentation skills. These sections also use a four-point scale. Finally, the Objective Data section compiles the grade obtained for the project (ranging from 0 to 10) and the days elapsed from the project assignment to its oral defense (project duration). The final ver-sion of the questionnaire was reviewed by the aforementioned committee of experts, who confirmed that the items would be interpreted as intended.

A written introduction to the questionnaire helped the participant supervisors to in-terpret correctly the items and their scales. This introduction explained the objective of the study, the structure of the questionnaire, and the scale used. It also included examples to illustrate the intended interpretation of the scales, as described in a previous paragraph. Finally, we added an acknowledgement of the professor’s contribution. The entire ques-tionnaire took about 10 minutes to complete.

Analysis of the data combined two multivariate statistical methods. First, a factor analysis reduced the initial set of aspects covered in the questionnaire to a set of essential functional factors of supervision. Second, a cluster analysis on these factors determined the supervision typology. One advantage of this combined technique is the reduction of the initial pool of variables to a smaller and more comprehensible set of factors, facilitating the cluster determination and interpretation. However, the factor computation eliminates a percentage of variance that, in some cases, could be determinant for the clustering. The cluster analysis can also be conducted on the original group of variables to avoid this loss (Dolnicar & Grun, 2008), but the high number of variables could make the cluster inter-pretation very difficult; in our case this would have required studying 27 different aspects for each cluster. To take advantage of both approaches, we performed the cluster analysis twice, first on the factor dimensions and second on the original set of variables. After that,


we compared the results and only considered the individuals classified under the same ty-pology through both analyses.

RESULTS

SampleThe sample was taken from capstone project advisors for a computer science engineering degree program. We gathered surveys from them throughout the past two years. The group consisted of 55 advisors supervising 232 projects. In all, 109 surveys were success-fully collected.

Preliminary StudyPrior to the proposed research we studied possible correlations between two of three basic measures of a capstone project: the overall supervision time, the grade obtained, and the project duration. The first presents data from the perspective of su-pervision, whereas the latter two constitute the principal objective data. We did not find any significant correlation (indeed, all the correlation coefficients had an absolute value of less than 0.1). Therefore, we discarded any simplistic assumptions for supervi-sion success, such as that more advisor effort leads to a better grade.

Identifying the Factor StructureBefore identifying the factor structure, it is important to analyze the consistency and integrity of the survey results. To this end, we studied both the reliability and the cri-terion-related validity of the first section of the questionnaire (items 1 to 26, because we excluded the global measure item). Reliability assesses the internal consistency of the items, whereas criterion-related validity refers to their concurrent validity (Hair, Black, Babin, & Anderson, 2009). The instrument has a high reliability with a Cron-bach’s alpha of 0.865. Regarding the second facet, we studied the correlation between the global measure item and the score obtained summing the other 26 items. We also compared this sum with the overall time spent by the advisor, as declared in the sec-ond questionnaire section (assuming that more involvement would correlate with more time spent). Positive correlations of 0.719 (p � 0.001) and 0.545 (p � 0.001) were obtained, respectively, representing an acceptable criterion-related validity (Hair et al., 2009).

We then performed two tests commonly done prior to factor analysis. First, Barlett’s sphericity test obtained a chi-square of 1632.51 (gl � 325, p � 0.001), which suggests that the intercorrelations matrix contains sufficient common variance to make this analysis appropriate. Second, the Kaiser-Meyer-Olkin measurement of 0.709 indi-cates the sampling adequacy for a factor analysis to proceed (corresponding to values greater than 0.5) (Hair et al., 2009).

Then, we examined the data sample of 109 cases, employing a principal components analysis as the extraction technique, and varimax as the orthogonal rotation method. A minimum eigenvalue of 1 was used as the cut-off value for extraction. Table 1 summariz-es the variables’ factor loadings. It contains seven factors, which explain 70.24% of the total variance. All the variables have a factor loading larger than 0.5 in the selected factor, usually considered excellent. Furthermore, there are no factor loadings larger than 0.5 in the other factors. This fact demonstrates the absence of multifactorial items (Hair et al.,


TABLE 1 Factor Analysis Results

Factor Reliability Item Loading

Corrected item-to- total

correlation

Technology 0.809

I took part in the choice of the development technology

0.838 0.548

I stated the quality to be reached by the project deliverables (security level, performance...)

0.700 0.705

I took part in the choice of the design technologies (architecture, patterns…)

0.624 0.658

Faced with technical errors, I was involved in their correction

0.600 0.627

Arrangements 0.789

I was involved in the time estimation of the project tasks (WBS)

0.846 0.738

I was involved in developing the project schedule (Gantt).

0.840 0.721

I was involved in the preparation of the oral presentation (attending rehearsals, advising)

0.598 0.465

I took part in the choice of the development methodology

0.512 0.520

Keepalive

0.766

I took the initiative in keeping alive the contact with the student

0.796 0.632

Faced with delays in message answering, I insisted

0.726 0.630

I took part in deciding the project closing 0.719 0.451

Faced with dead times, I started reactivation actions

0.586 0.589

I monitored the final deadline fulfillment 0.526 0.392

Execution 0.764

Faced with development problems whose solution I knew, I was involved in their resolution

0.807 0.581

Faced with development problems whose solution I did not know, I was involved in their resolution

0.721 0.636

I changed my supervision style according to the project evolution

0.694 0.478

Faced with management problems, I was involved in their correction

0.555 0.598

Meetings 0.720

I was involved in setting up the project meetings (scheduling, contents, writings to review...)

0.847 0.571

Faced with inadequately organized meetings, I started corrective measures

0.749 0.528

I took part in the choice of the meetings frequency

0.555 0.459

I took part in the choice of the meetings format (formal aspects, duration, minutes…)

0.520 0.483

Continued


Factor Reliability Item Loading

Corrected item-to- total

correlation

Management 0.713

I took part in deciding when to reschedule the project

0.795 0.533

I was involved in the project management

0.641 0.560

I monitored the schedule fulfillment (Gantt) during the project execution

0.637 0.536

Reports 0.620

I was involved in the final report writing

0.830 0.459

I was involved in the project defense presentation document elaboration

0.727 0.459

Note: Each item has a loading larger than 0.5 in the factor where it has been located as shown in the table. The loading of each item in the other factors is not greater than 0.5 in all the cases.

TABLE 1 Continued

2009; Wang, Wang, & Shee, 2007). The table also includes the variables’ corrected item-to-total correlation and the internal reliability of each factor. Both criteria illustrate the high consistency of the obtained factor structure. With regard to corrected item-to-total correlation, this factor is lower than 0.4 in only one case; however, some studies do con-sider this an acceptable value (Merckelbach, Horsellenberg, & Muris, 2001). With re-spect to factor reliability, only the last factor is lower than 0.7, the minimum standard suggested for basic research, and also greater than 0.6, usually admitted for exploratory factor analysis (Hair et al., 2009).

The next stage consisted in assigning each factor a label, which captures the essence of the items included in the factor. We proposed the following labels and descriptions:

Technology The degree to which the advisor intervenes in the technological aspects of the project. This factor encompasses both specialized technological assistance and technological decision support.

Arrangements The degree of participation in preparing for the project. This factor covers the planning and initial decision making prior to project execution as well as assisting the student in the groundwork for the oral defense.

Keep-Alive The degree of effort devoted to maintaining the student active in the project. This aspect includes reacting to downtimes in the project evolution and maintaining contact with the student.

Execution The degree of involvement over the course of the project. The main activity is support for nontechnical problems encountered throughout the project’s development.

Meetings The degree of involvement in meeting with the student. This facet includes the organization and holding of meetings.

Management The degree of support in project management. This factor considers management support activities that follow the project scheduling.


Reports The degree of intervention in the realization of the project’s final deliverables. This aspect concerns the final report and the presentation document for the oral defense.

The questionnaire’s second section consists of four items concerning the time devoted by the advisor to meetings, communication with the student, revision of reports, and overall time. Item 31 was previously compared with the sum of the first 26 items from the first sec-tion. We also compared the other items from the second section with their potentially relat-ed factors, again under the assumption that more time spent corresponds to more involve-ment. We obtained a significant positive correlation between the meetings factor and time devoted to meetings (� � 0.320, p � 0.01). The same occurred in the case of the keep-alive factor and time invested in communication (� � 0.227, p � 0.05). For the reports factor and time spent reviewing reports, there was a positive correlation; but it was not statistically significant (� � 0.152, p � 0.114).

Identifying the Cluster StructureCluster analysis is a common term for a variety of multivariate methods that classify indi-viduals into groups whose pattern scores on variables are similar (Hair et al., 2009). At this point, we used the seven factors generated previously as categorization items in a cluster analysis, with the goal of determining a supervising typology. Therefore, the original 26 four-point Likert-type items for each case have been substituted here for seven z-scores (with mean 0 and variance 1).

In the first stage, we applied a hierarchical technique, the Ward method with squared Euclidean distance, to establish the number of clusters and to identify outliers. For this task, we used the dendrogram analysis technique. In the second stage, we em-ployed a k-means clustering to refine the initial groups (Hair et al., 2009; Wang & Liu, 2008).

When we analyzed the dendrogram produced by the Ward method, we could recog-nize between five and seven clusters without individual outliers. A set of seven clusters produced a pair with very similar interpretation. With five, one of the clusters integrated two very different groups. Therefore, we concluded that the most appropriate number was six. Using the final centroids produced by this method as the initial centroids for a k-means clustering, we obtained the set of centroids shown in Table 2. This table also in-cludes the number of cases in each cluster.

We repeated the analysis on the original variables and compared the results. We also found a six-cluster structure with a coincidence of 78.9% (86 cases in the same place). The kappa index of agreement between both cluster analyses was 0.726 (p � 0.001), considered a high level of agreement (Strijbos, Martens, Prins, & Jochems, 2006). The clusters shared 31, 26, 10, 9, 5, and 5 cases, respectively, with 3 and 4 being the clusters that lose more cases. For the remainder of the research, we only considered the shared cases in the statistical data analyses. The lost cases can be interpreted as lo-cated at the border of two clusters (each one from a different analysis), so we discarded them.

Figure 2 graphically reflects the contents of Table 2 and shows the six cluster spider graphs. In each diagram we present the score obtained in the cluster for each factor. Because these values are z-scores, zero represents the mean of all the cases, and val-ues �0.5 or greater describe whether a dimension was scored relatively “high” or “low” compared with the rest (Wang & Liu, 2008).


The next stage consisted of interpreting each cluster, where each cluster corresponds to a supervision style. We propose the following style names and descriptions for the most representative factors of each cluster (see Figure 2):

Student alone In this supervision type, the advisor delegates most of the project tasks to the student. The arrangements factor is above the average, showing that the student receives assistance with planning and the oral defense preparation. The rest of the dimensions are below the mean, reflecting either average or little advisor contribution.

Execution focused The most prominent features of this style are the large amount of support during the project execution and, conversely, the little importance given to planning, oral defense preparation, and final report elaboration. This type of scoring in the factors could indicate that the advisor accepts the student’s proposed schedule and, deeming it adequate, provides assistance in problem resolution during the project execution. With this support, the student is expected to write a good report and accomplish an adequate oral defense without assistance. Another notable facet is the involvement in the organization of meetings, the tool most likely used for problem-solving support.

Global supervision In this supervision type, the advisor is mainly concerned with advancing the project and ensuring that the student makes the final report and the presentation document for the oral defense. The advisor is involved in keeping the project alive, as well as in monitoring its execution, though to a lesser extent than the previous style. Because every factor is above average, this style may be perceived as a global supervision.

Management focused The main concern here is project management, which comprises monitoring schedule compliance and influencing rescheduling decisions. Meetings with the student could be used to this end. However, the student obtains little support for other activities, except for a little help for writing reports. So, it seems that the advisor assumes the proposed planning is appropriate and leaves the execution and technology aspects in the student’s hands.

TABLE 2 Classification of Supervision Styles (k-means Final Cluster Centroids)

Factor Cluster 1 Cluster 2 Cluster 3 Cluster 4 Cluster 5 Cluster 6

Technology –0.544 0.013 0.207 –0.396 2.657 0.958

Arrangements 0.408 –0.502 0.283 –0.768 –0.692 2.245

Keep-Alive –0.098 –0.277 0.807 –0.207 –0.756 0.551

Execution –0.616 0.998 0.575 –0.685 –0.558 −0.500

Meetings –0.615 0.269 –0.042 0.483 –0.948 2.396

Management –0.544 –0.193 0.141 1.369 –0.709 0.374

Reports –0.120 –0.680 1.037 0.300 0.426 –1.189

n 34 27 20 17 6 5


Technological mentoring The focus here is almost exclusively on technological aspects, including the selection of technologies. The suitability testing of new technologies for specific problems is a typical example of this kind of supervision. Thus, the advisor’s purpose could be to discover the benefits and drawbacks of the chosen technologies as revealed through the student’s experience with them, which is expressed in the final project report. This focus explains the advisor’s considerable involvement in the reports factor that characterizes this style. On the other hand, this supervision style shows very little involvement in the other factors. It is noteworthy that here we find the lowest levels of contribution to meeting organization, management, and keeping the project alive.

FIGURE 2. Cluster spider graphs representation with factor z-scores of Table 2.


Process focused This style emphasizes the process of software development and tasks such as planning, oral defense preparation, and meeting organization. The advisor is also very concerned with technology, keeping the project alive, and project management. Almost all of these factors are common to any kind of project. In contrast, the execution and reports factors, which depend on the specific project, obtain very little assistance. The advisors of this cluster probably consider that the strong support provided in software development is enough to allow students to achieve a good product (software and documentation) on their own.

Complementing the Styles CharacterizationIn the following, we analyze data from sections two, three, and four of the questionnaire to draw further conclusions that confirm and complement our initial interpretation of the six styles.

Analysis of time devoted by the advisor Table 3 gives the advisor-devoted time scores grouped by style, including meetings, communication with the student, revision of reports, and overall time. These data give the advisor’s perception of the effort made during capstone project supervision. Significant differences were obtained between the styles in each of the four variables. The process focused and the global supervision styles indicated the most time invested. In contrast, the management focused and the student alone styles had the smallest amounts of time. For the sake of simplicity, in the following we consider only the overall time because it sums up the other variables. We will refer to this variable as mean advisor’s devoted time.

Analysis of student results Table 4 presents the mean grade obtained in each style and the average number of days elapsed from the project assignment to the oral defense (project duration). There were significant differences among the styles regarding the grade parameter and merely a tendency of significant differences in the duration parameter. It should be noted that students with the global supervision style obtained the lowest grades, whereas they devoted the most amount of time. On the other hand, the technological mentoring style had the highest grades, and the execution focused style, the least amount of time (although it was similar to student alone, technological mentoring, and process focused styles).

Student classification according to skills Table 5 classifies students from their advisors’ viewpoint. This classification was obtained through a second cluster analysis based on the student skills questionnaire section. Here, we followed the same steps as in the previous cluster analysis. Since there are two types of students with a difference of at least one point between the cluster centroids in each variable, we interpret the two clusters as the better skilled students and the worse skilled students. There are significant differences between both clusters considering the grade and the amount of days invested by the student. The better skilled students received a mean (SD) of 8.9 (0.92) points, whereas the remainder obtained 7.76 (1.47) points (U � 467; p � 0.001). On the other hand, the better skilled students spent a mean (SD) of 467 (219) days, whereas the rest of the students devoted 607 (309) days (t � 2.394; p � 0.05). Conversely, there is no


significant difference in the advisor’s devoted time (U � 767; p � 0.307). The better skilled students had a mean (SD) of 2.72 (0.82), which means close to “quite a lot” of advisor attention, whereas the worse skilled students received 2.55 (0.79). Hence, the worse skilled students obtained lower grades, invested more days, and received similar advisor attention as compared to the better skilled.

Table 6 shows student distribution according to the student and supervision types. There are significant differences in the distribution of the students on this table (�2 � 11.749, df � 5, p � 0.05). The global supervision and management focused styles had a higher percentage of worse skilled students. On the other hand, the technological men-toring and process focused styles were mostly applied to better skilled students. However, this result should be considered carefully because several cells contain fewer than five cases.

TABLE 3Time Devoted by the Advisor Categorized by Supervision Style (Mean and SD)

Task

Style 1Student

alone

Style 2Execution

focused

Style 3Global

supervision

Style 4Management

focused

Style 5Technological

mentoring

Style 6Processfocused

Statistical test

Meetings1.9

(0.6)2.69

(0.74) 3 (0.82)

2.44 (0.88)

2.8 (1.1)

4 (0)

a�2 � 31.474,

gl � 5***

Commu-nication

2.06 (0.68)

2.5 (0.65)

3 (0.82)

1.89 (0.6)

2.8 (1.1)

2.4 (0.59)

a�2 � 15.274, gl � 5**

Reports2.29

(0.69)2.65

(0.56)3.4

(0.7)2.22

(0.44)3.2

(0.84)4

(0)

a �2 � 32.284, gl � 5***

Overall time

2.23 (0.67)

2.73 (0.6)

3.4 (0.52)

2.11 (0.33)

3 (1.23)

4 (0)

a �2 � 36.002, gl � 5***

n 31 26 10 9 5 5

Note: aKruskal-Wallis test; **p � 0.01; ***p � 0.001.

TABLE 4Evaluation of Each Supervision Style by Grade and Days Spent (Mean and SD)

Parameter

Style 1Student

alone

Style 2Execution

focused

Style 3Global

supervision

Style 4Management

focused


mentoring


Statistical test

Grade8.66

(1.03)8.42

(1.15)7.1

(1.77)8.5

(1.15)9.6

(0.22)8.9

(1.43)

a�2 � 13.446, gl � 5*

Project duration (days)

485(293)

459(207)

721(256)

655(317)

475(128)

465(93)

bF � 2.167, p �0.066

n 31 26 10 9 5 5

Note: aKruskal-Wallis test; bANOVA test; *p � 0.05.


Summary of the data analysis of the identified styles Table 7 summarizes the previ-ously studied parameters regarding style. In order to simplify interpretation, we represent the goodness of a parameter for a given style with the symbol for good, x for bad, and a blank for medium. More specifically, we label as good (or bad) those re-sults that deviate from the arithmetic mean by plus (or minus) 0.5 standard derivations. Otherwise, we consider the results as medium. For the advisor-devoted time and project duration variables, we applied the reverse approach, i.e., the lower the value, the better the qualification. To label the student type we have considered the better skilled students’ percentage in the style.

A preliminary examination of Table 7 reveals that the pair of Styles 1 and 2 shares the same qualifications for student type and project duration. The same happens with Styles 3 and 4, and for Styles 5 and 6. Nevertheless, there are differences in the other two parameters. An analysis of this table allows us to elaborate on our supervision typology:

1. The student alone style scores well in the advisor-devoted time parameter. This fact agrees with the name chosen for this style. The projects here had a good duration and an average grade.

2. The execution focused style achieved a good project duration and average scores in the remaining parameters. We observed more advisor-devoted time in this style than in the student alone style, with equal results in grade and duration, and for students of similar type. Furthermore, considering the data in Table 4, it is

TABLE 5Classification of Students by Skill (k-means Final Cluster Centroids)

Student skill Cluster AWorse skilled students

Cluster BBetter skilled students

Autonomy 2.417 3.426

Management skills 1.729 3.147

Technology and methodology skills 2.167 3.639

Meetings and communication skills 2.083 3.443

Writing skills 2.208 3.393

Public presentation skills 2.542 3.557

n 33 (38.4%) 53 (61.6%)

TABLE 6Distribution of Student and Supervision Type

Supervision styleCluster A

Worse skilled studentsCluster B

Better skilled students

Student alone 10 (32%) 21 (68%)

Execution focused 9 (35%) 17 (65%)

Global supervision 7 (70%) 3 (30%)

Management focused 6 (67%) 3 (33%)

Technological mentoring 1 (20%) 4 (80%)

Process focused 0 (0%) 5 (100%)

n 33 (38%) 53 (62%)


noteworthy that this style produced more uniform project durations than the student alone style (smaller standard deviation of project duration).

3. The global supervision style presented the worst scores. It had longer-than-average duration and bad grades. The large amount of time devoted by the advisor did not improve student results. The elevated percentage of answers (90%) with high scores for the item “I changed my supervision style according to the project evolution” suggests that the advisor adopted this style as a reaction to the student’s apparent passivity. The high score in the keep-alive dimension of this style (see Figure 2) seems to confirm this idea.

4. The management focused style, as with the previous one, was used especially with those worse skilled students, who tended to lengthen their projects over time. However, and unlike the global supervision style, the grade obtained was average, and the keep-alive factor (see Figure 2) was below average. The low advisor-devoted time is consistent with our previous style interpretation: the advisor mainly supervises the planning compliance but delegates the project execution to the student.

5. The technological mentoring style fostered short-duration projects together with the best grades. The technological focus could explain the high project quality. The use of recent and lesser-known technologies probably produced more interesting, difficult, and therefore more valuable projects, according to the evaluation guide mentioned in the Introduction. This supervision type was mainly applied to better skilled students. This is consistent with our previous interpretation of the style, since strong students are likely to choose projects in which they can study the suitability of new technologies.

6. The process focused style had the highest advisor-devoted time, which denotes a great advisor effort throughout the project. As in the technological mentoring style, the students were mainly better skilled and the projects were completed in a relatively timely fashion. This style featured the most uniform project duration when compared to the other styles (the smallest standard deviation); it did not, however, produce good grades. Advisor effort is mainly devoted to process-related tasks such as planning, meetings, or management. This focus could contribute to shortening the project duration, but without providing any added value according to the evaluation guide.

TABLE 7Summary of Style Parameters

Parameter

Style 1Student

alone

Style 2Execution

focused

Style 3Global

supervision

Style 4Management

focused


mentoring


Student type x x

Project duration x x

Grade x

Advisor’s devoted time

x x


DISCUSSION

Capstone project supervision is a multifaceted commitment (Clear et al., 2001). The seven factors proposed in our study can be considered the main supervision duties and are connected with the five basic process groups of the PMBOK® Guide (Project Man-agement Institute, 2008). Three of these, planning, executing, and monitoring and con-trolling, are strongly related to the arrangements, execution, and management factors, re-spectively, of this study. The other two, initiating and closing, are included in the arrangements and reports factors, correspondingly. Additionally, we obtained the meet-ings and keep-alive factors, which are more specific advisor tasks, as described by other authors (Clear et al., 2001; Joy, 2009; Bouki, 2007; Malik et al., 2009).

Some authors mention supervision roles instead of styles. Clear et al. (2001) distin-guish among monitor/observer, director, manager, sponsor, teacher, mentor, and tech-nical guru. Fincher et al. (2001) propose observer/commentator, line manager, project manager, and master/mentor. Joy (2009) simplifies the last four as administrator and manager. Scott (2008) compares a capstone project advisor with a coach who should assume three main responsibilities: mentor, mediator, and manager. Finally, James et al. (2005) consider the personal aspect of supervision to be of great importance, men-tioning roles such as pastoral care. Without a suitable procedure to match the suggest-ed roles and our studied styles, only an intuitive relation can be established between them. For instance, our student alone style and the monitor/observer role proposed by Clear et al. (2001) seem to be similar ideas, as are technological mentoring and techni-cal guru. However, the global supervision style, for example, can be interpreted as a mixture of director (making execution decisions), teacher (providing timely informa-tion), and perhaps some pastoral guidance (advising the student on how to proceed and how to finalize the project as soon as possible).

In the literature it is difficult to find a clear definition of successful student and advi-sor work in capstone projects. Wieck (2003) defined it in terms of how well the project prepares students for industry. Janicki, Fischetti, & Burns (2007) suggest that the de-gree of practical application of the developed product can be considered a measure of project effectiveness. Our approach was similar to that of Delany (2008), based on the time spent by the student and advisor and the grade obtained for the project.

The literature reflects a lack of agreement regarding the purposes of feedback. For ex-ample, Shute (2008) describes its two main functions as directive versus facilitative. The former tells students what needs to be fixed or revised, whereas the latter guides students in their own revision. Another interesting question regarding feedback is when to provide it. In the case of capstone projects, it could be important to provide more intense feedback at critical times; yet at other times, the advisor should allow the student to advance inde-pendently. This interesting aspect not covered by the present study will be addressed in our future work.

This study has limitations to be tackled by future research. For example, the sample could be extended in order to enhance the validity of the results. The study could also be conducted at additional universities and in additional countries to determine whether the results are suitable for these different environments. In addition, it could be interesting to study whether the results are applicable to other engineering disciplines. This study has considered exclusively the advisor’s opinion, but the student’s viewpoint should be also taken into account. It would be interesting to analyze how student type influences the


advisor’s supervision style, and which style is more effective in each case. Furthermore, other features related to supervision style could also be analyzed, such as the variation in style of an individual advisor with different student or project type, the quality of the time devoted by the advisor and student, and the variability of project scope. Finally, now that the styles have been determined, it may be interesting to study how advisors with con-trasting styles supervise their students by monitoring the advisor-student interaction in each case. This process could be performed using a “think-aloud protocol” to determine the methodology followed in each case.

CONCLUSIONS

By analyzing the level of involvement of advisors in different functional duties, this study developed and validated an instrument to determine supervision styles applied by cap-stone project advisors in the area of computer science engineering.

To determine the styles, we first researched the factors that constitute them. To this end, we produced a questionnaire and compiled more than 100 surveys. After performing an exploratory factorial analysis, we detected seven relevant factors. We called them tech-nology, arrangements, keep-alive, execution, meetings, management, and reports depend-ing on the main features comprising each one. Next, these factors were used as classifica-tion items in a cluster analysis that identified six main styles of supervision, which we named student alone, execution focused, global supervision, management focused, tech-nological mentoring, and process focused. The questionnaire also contained sections de-voted to the time invested by the advisor, the student’s perceived skills, and selected objec-tive data. These data were used to elucidate the six supervision styles and reveal differences between them.

The supervision styles identified and characterized in our research can help advisors recognize ways to proceed in the supervision of capstone projects, and enable them to im-prove their work by making better-informed decisions on the basis of approaches that have been tested by other advisors. The resulting style descriptions could also be useful for advisor training courses.

ACKNOWLEDGEMENTS

We wish to thank the experts committee for their help in the questionnaire refinement, the paper referees for their valuable comments, and finally all the survey participants for their collaboration.

This work was partially supported by Universidad de La Rioja-BSCH [projects API11/08 and API11/11].

REFERENCES

ACM/IEEE-CS. (2001). Computing curricula 2001: Computer science. Retrieved from http://www.acm.org/education/education/education/curric_vols/cc2001.pdf.

ACM/IEEE-CS. (2008). Computer science curriculum 2008: An interim revision of CS 2001. Retrieved from http://www.acm.org//education/curricula/ComputerScience2008.pdf.

Armstrong, S. (2004). The impact of supervisors’ cognitive styles on the quality of research su-pervision in management education. British Journal of Educational Psychology, 74, 599– 616.


Bouki, V. (2007). Undergraduate computer science projects in UK: What is the point? Pro-ceedings of the Informatics Education Europe II Conference IEEII 2007, 176–183.

Clark, M.A.C., & Boyle, R.D. (1999). A personal theory of teaching computing through final year projects. Computer Science Education, 9(3), 200–214.

Clear, T., Goldweber, M., Young, F.H., Leidig, P.M., & Scott, K. (2001). Resources for in-structors of capstone courses in computing. SIGCSE Bulletin, 33(4), 93–113.

Delany, D. (2008). A review of the literature on effective PhD supervision. Trinity College, Dublin, Centre for Academic Practice and Student Learning (CAPSL).

Dolnicar, S., & Grun, B. (2008). Challenging “Factor Cluster Segmentation,” Journal of Trav-el Research, 47(1), 63–71.

Donnelly, R., & Fitzmaurice, M. (2009). Resource pack on supervising postgraduate students. Dublin Institute of Technology. Retrieved from http://www.dit.ie/lttc/media/ditlttc/doc-uments/Microsoft%20Word%20-%20Supervising%20Postgraduate%20Students.pdf

Dutson, A.J., Todd, R.H., Magleby, S.P., & Sorensen C.D. (1997). A review of literature on teaching engineering design through project-oriented capstone courses. Journal of Engi-neering Education, 86(1), 17–28.

Farr, J.V., Lee, M.A., Metro, R.A., & Sutton, J.P. (2001). Using a systematic engineering de-sign process to conduct undergraduate engineering management capstone projects. Jour-nal of Engineering Education, 90(2), 193–197.

Fincher, S., Petre, M., & Clark, M. (2001). Computer Science Project Work: Principles and Prag-matics. London: Springer.

Fraser, R., & Mathews A. (1999). An evaluation of the desirable characteristics of a supervi-sor. Australian Universities’ Review, 42(1), 5–7.

Gatfield, T. (2005). An investigation into PhD supervisory management styles: Development of a dynamic conceptual model and its managerial implications. Journal of Higher Educa-tion Policy and Management, 27(3), 311–325.

Greenbank, P., Penketh, C., Schofield, M., & Turjansky, T. (2008). The undergraduate disser-tation: ‘most likely you go your way and I’ll go mine.’ The International Journal for Quality and Standards, 2(1), 139–162.

Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2009). Multivariate Data Analysis. 7th ed. New Jersey: Pearson.

Ho, B. (2003). Time management of final year undergraduate English projects: Supervisees’ and the supervisor’s coping strategies. System 31(2), 231–245.

James, H.A., Hawick, K.A., and James, C.J. (2005). Teaching students how to be computer scientists through student projects. Proceedings of the 7th Australasian conference on Comput-ing education, 42, 259–267.

Janicki, T.N., Fischetti, D, & Burns, A. (2007). Incorporating real world projects and emerg-ing technologies into one MIS capstone course. Information Systems Education Journal, 24(5), 1–6.

Joy, M. (2009). Undergraduate computing projects – an investigation into the student experi-ence. ITALICS, 8(1), 60–78.

Keogh, K., & Venables, A. (2009). The importance of project management documentation in computing students’ capstone projects. Asia-Pacific Journal of Cooperative Education 10(3), 151–162.

Lee, A. (2010). New approaches to doctoral supervision: Implications for education develop-ment. Educational Developments, 11(2), 18–23.


Malik, M., Khusainov, R., Zhou, S., & Adamos, V. (2009). A two year case study: Technology Assisted Project Supervision (TAPaS). Engineering Education, 4(2), 76–83.

Marin, J.A., Armstrong, J.E., & Kays, J.L. (1999). Elements of an optimal capstone design experience. Journal of Engineering Education, 88(1), 19–22.

Merckelbach, H., Horsellenberg, R., & Muris, P. (2001). The Creative Experiences Ques-tionnaire (CEQ): A brief self-report measure of fantasy proneness. Personality and Indi-vidual Differences, 31, 987–95.

Pressman, R.S. (2010). Software engineering: A practitioner’s approach. 7th ed. New York: McGraw-Hill Higher Education.

Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK Guide). 4th ed. Newton Square, Project Management Institute.

Scott, H. (2008). Management of real-world projects in university computing courses. In Proceedings of Student-Faculty Research Day, CSIS, Pace University, May 2nd 2008 (pp. A2.1–A2.7).

Strijbos, J., Martens, R., Prins, F., & Jochems, W. (2006). Content analysis: What are they talking about? Computers & Education, 46, 29–48.

Shute, V.J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

Todd, M.J., Smith, K., & Bannister, P. (2006). Supervising a social science undergraduate dis-sertation: Staff experiences and perceptions. Teaching in Higher Education, 11(2), 161–173.

Wang, C.K.J., & Liu, W.C. (2008). Teachers’ motivation to teach national education in Sin-gapore: A self-determination theory approach. Asia Pacific Journal of Education, 28(4), 395–410.

Wang, Y.S., Wang, H.Y., & Shee, D.Y. (2007). Measuring e-learning systems success in an organizational context: Scale development and validation. Computers in Human Behavior, 23, 1792–1808.

Wieck, M. (2003). The capstone project – A foundation for work? Proceedings of 16th Annual Conference, National Advisory Committee on Computing Qualifications. Palmerston North, New Zealand, 455–458.

AUTHORS

César Domínguez Pérez is an assistant professor at the Mathematics and Computer Science Department of the Universidad de La Rioja, C/ Luis de Ulloa, s.n., Logroño, La Rioja, 26004 Spain; [email protected].

Arturo Jaime Elizondo is an assistant professor at the Mathematics and Computer Science Department of the Universidad de La Rioja, C/ Luis de Ulloa, s.n., Logroño, La Rioja, 26004 Spain; [email protected].

Francisco J. García-Izquierdo is an assistant professor at the Mathematics and Com-puter Science Department of the Universidad de La Rioja, C/ Luis de Ulloa, s.n., Lo-groño, La Rioja, 26004 Spain; [email protected].

Juan José Olarte Larrea is an assistant professor at the Mathematics and Computer Science Department of the Universidad de La Rioja, C/ Luis de Ulloa, s.n., Logroño, La Rioja, 26004 Spain; [email protected].

supervision typology in computer science engineering capstone projects

Documents