THE PROMOTION AND ASSESSMENT OF GENERIC SKILLS FROM INTERDISCIPLINARY TEACHING TEAMS

Anna Carreras-Marín, Yolanda Blasco Martel, Marc Badia-Miró, Manuela Bosch-Príncep, Isabel Morillo-López, Gemma Cairó-i-Céspedes, Maria Dolores

Casares Vidal Universitat de Barcelona (SPAIN)

annacarrerasmarin@ub.edu, yolandablasco@ub.edu, mbadia@ub.edu, manubosch@ub.edu, imorillo@ub.edu, gcairo@ub.edu, lola.casares@ub.edu

Abstract The paper explains a teaching project financed by the University of Barcelona (UB). It focuses on a generic skill of the University of Barcelona, which is defined as "the learning capability and responsibility”, and in which analytical and synthesis skills are included. It follows a multidisciplinary approach including teachers of Mathematics, World Economics and Economic History. All of us share the same students during the first and the second course of the grade in Economics at the Faculty of Economics and Business. The project has been developed in three stages. The first one has been done during the first semester of the course 2012/13, being applied to first year students on the subjects of Mathematics and Economic History. The second phase is being to be done during the second semester only on the Economic History subject. A third stage is going to be done next course 2013/14 to second year students on the subject of World Economics. Each different teaching team has developed specific materials and assessment tools for each one of the subjects included in the project. The project emphasizes two teaching dimensions: the elaboration of teaching materials to promote the acquisition of generic skills from an interdisciplinary point of view, and the design of specific tools to assess such skills. The first results of the first phase of the project shows clear deficiencies in the analytical skill regarding to first year students.

Keywords: Innovation, Learning Experiences in Higher and Further Education, General Skills.

1 INTRODUCTION The teaching innovation project presented in this paper focuses on a generic skill of the University of Barcelona (UB), which is defined as "the learning capability and responsibility”. It is composed by the analytical skill, which is the ability to visualize, gather information, articulate, analyze, solve complex problems, and make decisions. The promotion of analytical skills is included in the three subjects that we have selected to develop this teaching project: Mathematics, World Economics and Economic History. These subjects have a compulsory character into the Economy and Business and Management grades of our faculty. Although we have dealt with such different disciplines, we have used an interdisciplinary approach during the whole project.

As a previous step, we have identified a common topic which could be treated from a variety of approaches, which has been income inequality. Next step has been to elaborate teaching materials regarding to this topic, but specific of each one of our subjects. This work has allowed us to design three specific exercises around income distribution problems from the point of view of Mathematics1, World Economics2 and Economic History. The project has been developed in three stages. The first one has been done the first semester of the course 2012/13, being applied to the first year students on the subjects of Mathematics and Economic History. The second phase is being to be done the second semester only on the Economic History subject. A third stage is going to be done next course 2013/14 to the second year students on the subject of World Economics.

The project emphasizes two teaching dimensions: the development of teaching materials to promote the acquisition of generic skills from an interdisciplinary point of view, and the design of specific tools to assess such skills. The first results of the first phase of the project show clear deficiencies in the analytical skill regarding to the first year students.

1 See Manuela Bosch-Príncep & Isabel Morillo-López (2013) 2 See Cairó-i-Céspedes, Gemma & Lola Casares (2013)

Proceedings of EDULEARN13 Conference 1st-3rd July 2013, Barcelona, Spain

ISBN: 978-84-616-3822-20201

This project has been financed by the University of Barcelona (2012PID-UB/003) and some of the members of the team are involved in the following “Teaching Innovation Groups”: “GIDHEPSUB-Grup d’Innovació Docent Consolidat en Història Econòmica, Política i Social” and “EiA-MEFA- Noves metodologies per l'ensenyament i aprenentatge de la matemàtica econòmica, financera i actuarial".

2 METHODOLOGY

2.1 Transversal teaching teams: sharing training experiences from different disciplines

As a consequence of the implementation of the European Higher Education Area (EHEA), the University of Barcelona (UB) has developed a new set of regulations regarding to the promotion and assessment of general skills.3 According to that, general and transversal skills should involve the integrated use of knowledge, skills and attitudes. Consequently, without specific knowledge of each particular discipline, you cannot acquire the skills we are talking about. The promotion of the transversal skills is fully embodied in the very specific contents of each one of our subjects.

The teaching team, which has developed this project, includes teachers from Mathematics, World Economics and Economic History. Working jointly with such different disciplines is one of the main strengths of the project, although it also has raised some difficulties as well. Through the development of the project, we have learnt to combine different academic standards, non explicit rules of academic behaviour and even different types of organizing academic materials or ways of writing them. We have been forced to work interdisciplinary and transversal, as the first step to be able to training our students in such transversal skills.

Trying to find common places, we have identified a topic which could be treated from a diversity of approaches. It has been income inequality within countries and their analysis, a topic which its importance is increasing nowadays in the field of economics. The topic of income distribution allows us to use mathematical expressions to measure it, the use of long run data to analyse it through history and the theoretical economic framework to embody it in the world dynamics of the World Economics. We have created teaching materials regarding to this topic, but specific oriented to each one of our subjects. The final result has been the design of three specific exercises around income distribution measures from the point of view of Mathematics, World Economics and Economic History.

The three subjects being included into the project implement a continuous assessment system with a series of evaluative evidences to be collected throughout the course. Some of these activities are mandatory and some are optional. This common starting point has allowed us to propose a collaborative experience between subjects from different departments of the faculty. The common elements have been to share the same students of the first and second course of Economics at the Faculty of Economics and Business.

2.2 Pilot training experiences: acquisition of skills step by step According to AQU (2009) general skills can be acquired only at the very end of the whole training process. However, in our project we decided that in order to reach the highest level in the acquisition of skills, the learning process and the improvement of all of them, should start from the first course. The project has been developed in three stages according to three levels of acquisition of the skills from the first year grade. The first stage has been implemented the first semester of the course 2012/13, and it has been applied to the first year students on the subjects of Mathematics and Economic History. The second phase is being done the second semester only on the Economic History subject but to the same students that have previously been involved in the Mathematical experience. A third stage is going to be done next course 2013/14 to the second year students on the subject of World Economics, with the same group of students that we have worked with in the first year course.

In the first phase, the students have learned to measure the degree of income inequality based on the estimation of the functional form of a Lorenz curve for a particular population, within the Mathematics subject. In this exercise, most of the questions at the beginning of the exercise, have focused on the

3 Competencias transversales de la UB (2008), Normas reguladoras de la evaluación y de la calificación de los aprendizajes (2006), Normativa de los Planes Docentes (2012)

0202

understanding of the properties of the function, the mathematical calculations derived of it and as a consequence they have not been considered as an indicator of the analytical skill. It has been in the fourth question of the exercise that we faced the assessment of the skill. This question was devoted to test the availability of interpreting mathematically and, at the same time, understand the link of this result with the economy which was behind this result. This answer was weighted with 20% of the total score of the exercise. Additionally, we have also included some items regarding to other general skills, such as responsibility (if the student has accomplish each one of the deadlines for the activity) and teamwork skill (as the exercises were done in pairs). We have also measured it by self-assessment within each group.

We have also performed a complementary pilot experience in the course of Economic History with a different group of students, as a case control to compare the results with the other experiences. In this case, the students of the group that has made this activity are not expected to implement the subsequent project phases in the future. In that case we have used reading materials to analyse the implications of income inequality in many countries in the long run. In this case, we have taken into account more items in order to assess the analytical skill of the students. The exercise proposed had 3 questions, and each one of them included an analytical task. The share of this skill over the total score of the exercise has been higher in the Economic History experience, achieving 75% of the total, than in the Mathematical experience. In that case we have also included some items regarding to other general skills. Responsibility and written skills have been also taken into consideration weighting 10% on the score of the exercise. It is worth to say here, that in the case of the Economic History subject it has been more difficult to isolate the analytical skill from the contents of the subject, than it was in the Mathematics’ case.

In the second phase, during the second semester, we continue working the activity being done in phase 1 in the course of Economic History. In this case, the students should obtain a Lorenz curve and relate it with a reading of Branco Milanovic about economic income inequality4. To improve the analytical skills of the students, they are going to be asked to create their own learning schemes. In the next phase of the project, students will develop alternative measures of inequality (Gini index) in the subject of World Economics. In this third case, we are going to put more emphasis to the search and processing of economic databases.

The three project phases overlap each other, establishing three levels of competence acquisition. Although most of the activities are expected to be performed within the same group of students, we have been implementing complementary activities with different student groups although not involved in the previous or later phases of the project. This could help us to control the results derived from this project.

2.3 Promotion and assessment of general skills: rubrics as a training tool both for teachers and students

The rubrics have been defined as "a qualitative descriptor which establishes the nature of a performance" (Simon, 2001). According to Zazueta et al. (2008): "In the new paradigm of education rubrics are being used to give a more authentic or real value, to traditional qualifications expressed in numbers or letters, and it serves us to find out how the student is learning. The purpose is to improve the quality of teaching and learning, not to carry out an audit of them, instead of that it allows us to evaluate the process and the outcome."5.

From our point of view, this approach is particularly suitable for the assessment of competences, in which the qualitative component of the evaluation is particularly important. Additionally, the rubric has the advantage of being an instrument for the student to become aware of their strengths and weaknesses, which is a first step in the acquisition of skills. As it is going to be explained in the next section, one of the results of the present project points out to an overvaluation of the analytical skill by the students, one perception which is far from reality. As a consequence we should also include students’ perceptions on this question to be able to improve their general skills. This is one of the main reasons to use rubrics as an assessment tool.

In the first phase of the project we have designed two rubrics: one corresponding to the activity in the subject of Mathematics and the other for the Economic History course (see table 1). While the rubric of Mathematics was used both for teacher assessment and self-assessment by the students, the 4 Milanovic (2011). 5 Zazueta et al. (2008), p. 2

0203

Economic History tool was only used for the teacher evaluation of the activity. In the second phase, the previous rubric of the Economic History experience has been improved. We have also added a self-assessment task, similar to the one done in the Mathematics’ experience. Finally, the same methodology is going to be followed in the third phase of the project in which the students will also be evaluated and self-evaluated through a rubric.

Table 1- Rubric of the first experience in the subject of Economic History

3 RESULTS The first experience on the subject of Mathematics has been done by 70% of the students, due to the fact that it was a last optional exercise. The overall average score has been 7.38 over a maximum of 10. The students’ self-assessment was better than that, with 8.96. It indicates an optimistic bias of the students’ perception of their own work: an over-valuation of 15.8%, which we consider it is quite huge.

The worst result of the teacher assessment has been on the fourth question, the one counting for analytical skill (see table 2). Each one of the items shown in table 2, had been weighted differently in the final score of the exercise, in the following way: item 1 was 10%, item 2 was 30%, item 3 was 30%, item 4 was 20%, item 5 was 4%, item 6 was 4% and item 7 was 2%. Anyway, in the table we

0204

have represented the average results as shares over each one of the items, to be able to compare the different achievements of the students6.

As table 2 also shows, the largest deviation between teacher assessment and that of the student, it has been in the fourth question, which was the one focused on the analytical skill. In this particular question, the teacher assessment has been as an average of 0.75 over a maximum of 2; meanwhile the students’ self-assessment has been of 1.86. In this case, the optimistic bias increases to 55%, being it the highest discrepancy between both assessments.

Table 2- Results of the first experience in the subject of Mathematics

Items Nature of Indicators Teacher

Assessment

Students’

self-assessment

Discrepancies

1 Mathematical content 54.1% 89.7% +35.6%

2 Mathematical content 89.97% 84.94% -5.03%

3 Mathematical content 83.57% 90.40% +6.83%

4 Analytical Skill 37.5% 93.15% +55.65%

5 Responsibility 100% 100% 0%

6 Written Skill 72.5% 86.25% +13.75%

7 Teamwork Skill 98.5% 98% -0.5%

Source: Manuela Bosch-Príncep & Isabel Morillo-López (2013)

As it has been explained earlier, the Economic History experience has included more questions to assess the analytical skill. It has also included other general skills, like reading comprehension, responsibility and written skills. In this case, students’ self-assessment information was not provided. The exercise contained 3 questions. In each one of them an analytical task was performed by the students. Only 30% of the students had made the exercise, which implied 22 students. As in Mathematics’ experience, it was a last optional exercise performed in the course. The overall average score has been 6.1 over a maximum of 10 (see table 3). It should be emphasized that, again, the worst item has been in the analytical skill. It has had the lower score of all the items, with a 4.7 over a maximum of 10. This result is a bit better than that of Mathematics but it is already a bad result.

Table 3- Results of the first experience in the subject of Economic History

Items (aggregated results for the 3 questions) Average Teacher Assessment

Average overall result 6.1

Reading comprehension 7.3

Analytical skill 4.7

Responsibility 7.4

Written skill 9.7

The result of the assessment rubrics of the first phase of the project, both in the Mathematics and Economic History experiences, clearly indicates that the worst result obtained by students focuses on the indicator for the analysis skill. This result shows that the level of competition in analysis of first-year students of our Economic Grade is far below the rest of the skills considered or assimilation of

6 A more detailed explanation of these results are in Manuela Bosch-Príncep & Isabel Morillo-López (2013)

0205

the contents of the different subjects. It is consistent with those who argue that generic skills can only be assessed at the end of the training process, for example through the work of final grade. However, we believe that to achieve the best result in that final moment we have to start working skills from the first year.

The contrast of our results with the self-assessment surveys of students, in the case of the subject of Mathematics, has also enabled us to identify a negligible perception of this deficiency in the ability to analyze by the students. Thus, it is not only the item with the worst results in the assessments made by teachers, but also the item with greatest discrepancies between teacher evaluation and student self-assessment.

4 CONCLUSIONS After the pilot experiences of the first phase of the project, we can present our preliminary conclusions. The first results indicate a clear deficiency in the analysis skill in students of the first course in the grade of Economics. They are able to successfully perform exercises or repeat routine learning but they have serious difficulties in implement a reading analysis and also in applying their knowledge in different environments. The identification of this deficiency leads us to develop training methodologies to assess this skill in phase 2 and 3 of the project.

The first phase of the project has allowed us to establish a starting point in which, on the one hand, we have found the significant low level of our students in the analytical skill, and on the other hand, we found that the students' perception is far from reality in this regard. In the second phase of the project, which will be done in the second semester of 2012/13, the aim is that students will become aware of what is really their level of analysis and that they should improve it substantially. We have also made available to our students appropriate methodologies to enhance their competence, through the development of schemes summaries. The third phase will use these results to implement additional activities in the course of the second year, where the ability to analyze and synthesize reach higher levels of complexity.

The methodology followed in this project stresses the need to promote general skills gradually during the training process. It also reveals the importance of co-ordination among subjects in order to achieve an appropriate acquisition of general skills in our students. It is also worth to mention the need to develop transversal teaching materials to share common places among disciplines, such as the income inequalities materials we have used in our training experiences.

Although the project is limited to three specific subjects, its first results and conclusions, although preliminary, have interest beyond the particular cases of each one of the subjects. In this sense, it strengthens the arguments about the problems of working skills in isolation from each subject, risen the need to work through interdisciplinary teaching teams and in different levels of the training process, to adequately improve the generic skills of our students.

REFERENCES [1] AQU-Agencia Catalana per a la Qualitat Universitària (2009). Guia per a l'avaluació de

competències en l'àrea de ciències socials.

[2] Bosch-Príncep, Manuela & Isabel Morillo-López (2013). Evaluation of transversal competences: pilot project in a math course for first-year business degree students. Poster presentation at EDULEARN2013. Barcelona.

[3] Cairó-i-Céspedes, Gemma & Lola Casares (2013): Promotion of learning skills in the study of economics. Pilot project in the subject area of the World Economics. Oral presentation at EDULEARN2013. Barcelona.

[4] Competències transversals de la Universitat de Barcelona (2008). Col.lecció Normatives i Documents. Universitat de Barcelona

[5] Milanovic, Branko. 2011. “Más o Menos: La Desigualdad Del Ingreso Ha Aumentado En Los Últimos 25 Años, En Lugar De Disminuir Como Se Habia Previsto.” Finanzas y Desarrollo: Publicación Trimestral Del Fondo Monetario Internacional y Del Banco Mundial 48 (3): 6–11.

[6] Normativa dels Plans Docents (2012). Universitat de Barcelona.

0206

[7] Normes reguladores de l'avaluació i de la qualificació dels aprenentatges (2006) Universitat de Barcelona

[8] Zazueta Hernández & Herrera López (2008) “Rúbrica o matriz de valoración, herramienta de evaluación formativa y sumativa”. Quaderns Digitals / Quaderns número 55

0207