d 7 evaluation reporthistproject.no/sites/...report_from_qm_soft_final.pdf · do-it llp -project...

DO-IT LLP -Project number 2009-1-NO1-LEO05-01046 Version: 1.1 30.11.2011 of 60

DO – IT

Disseminating Open and Innovative Tools and Services for VET in quality Assurance

Transfer of Innovation Project2009-1-NO1-LEO05-010406

D 7 Evaluation Report


© Copyright 2011 DO-IT

Consisting of:1. Sør-Trøndelag University College (HiST), Trondheim, Norway2. Petrus Major University, Tirgu Mures, Romania2. BOKIK, Miskolc, Hungary3. University of Huddersfield, Huddersfield, UK4. QM Soft, Oslo, Norway

This document may not be copied, reproduced, or modified in whole or in part for any purpose without written permission from the DO-IT Consortium. In addition to such written permission to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced. All rights reserved.This document may change without notice, but consortium members should be informed, andnumber of version, stage and date should be given.

Legal notice:This project has been funded with support from the European Commission. This publicationreflects the views only of the author, and the Commission cannot be held responsible for anyuse, which may be made of the information contained therein.

Authors: Erik Engh, Quality Management Software as

Contributors: Joan Lu, Livu Moldovan, John B. Stav, Juhasz Gabor

Circulation: Open

Stage: 1.1


Executive Summary

A major difficulty for educational projects at an European level relates to the economical situation in the different countries. Another difficulty may be the in the communication of knowledge, know-how and competence. From rich oil-nations through to economical weak economies where the major topic is cost cutting in the public sector, it is easy to see that adequate funding secured in a project might be impossible to implement in a larger scale in the societies in real life.

DO-IT is a project that is relatively technological advanced from a hardware point of view and some of the key tasks and actions requires investments by local VET Schools and training institutions which, today, is not possible to reach.

Project promotion in the local societies are therefore difficult, if not impossible to implement, because the key elements can not be implemented within a realistic future with the current budget limitations. The results from the questionnaires reflects such observations.

The key topics in the project are, however, interesting.The national implementation for all topics in the project, is difficult or maybe not possible in a short time horizon.

The sustainability of the project will therefor also be fragmented, heavily depending on the economical development in the countries involved, and combined with the results and the activities each partner has carried out in the project period.

Testing of the SRS system, as a free standing system, has been a success in all cases where it has been used. It is not clear, from this project, how SRS will be supported and maintained for the institutions who will use it in the future. It is also a limited number of institutions that are able to carry out the investment in a high number of IPODS for implementation of the system.

ABT has been tested in different ways by the partners. However it seems that the understanding of what implementation of ABT really means for the practical implementation in a course structure differs. It is strongly recommended that a common understanding of such a central topic is ensured at the start of the project by all participants. A definition of what ABT is and how it should be implemented should have been defined early in the project.

The combination of SRS and ABT has not been tested thoroughly except for one training course at UTM where SRS has been implemented. On the other hand that training course do not follow the ABT methodology as originally defined in the referred project behind DO-IT, but it is pretty close.

The participants have all carried out courses and course activities. But through a far better coordination and harmonization of their activities the project results would have been better suited for later dissemination and exploitation.

The presentation of the project, on the project web sites appear as a Hist project. Missing references to partners web sites, missing logo for the partners etc makes the project appear as a single purpose project for Hist and not as an European cooperation project.

Conclusions and findings related to SRS are missing in this report. The required information has been promised by Hist, but not delivered within the promised time schedule.


Table of ContentsExecutive Summary.........................................................................................................................3Introduction......................................................................................................................................6Target Group for the project............................................................................................................ 9Definitions..................................................................................................................................... 10

Activity Based Training (ABT)............................................................................................10Quality Assurance.................................................................................................................10Quality Control.....................................................................................................................10Student Response System (SRS)..........................................................................................10ISO 9000...............................................................................................................................11ISO 3834...............................................................................................................................11ISO 19011.............................................................................................................................11Other production processes outside welding........................................................................12

What do we mean by Quality Assurance ?.................................................................................... 12Background....................................................................................................................................13Role of the project participants in the consortium.........................................................................15

HIST..................................................................................................................................... 15UOH..................................................................................................................................... 15UPM..................................................................................................................................... 15BOKIK................................................................................................................................. 16QMSOFT..............................................................................................................................16

Project elements.............................................................................................................................17Activity Based Training........................................................................................................17The Student Response System (SRS)...................................................................................18Quality Assurance.................................................................................................................19

The evaluation process...................................................................................................................19Objectives of this document.......................................................................................................... 20

Instructors, teachers and students.........................................................................................20The user market............................................................................................................................. 21

Stakeholders in the market................................................................................................... 22Methods employed for internal and external evaluation............................................................... 22Analysis and discussions............................................................................................................... 22

Evaluation.............................................................................................................................22Project objectives versus real activities................................................................................23User Requirements Report................................................................................................... 23Project fragmentation........................................................................................................... 23The training courses............................................................................................................. 24Instructors, teachers and students.........................................................................................25

Training and evaluation in Romania..............................................................................................25Training and evaluation in Hungary.............................................................................................. 28Training and evaluation in Norway............................................................................................... 29Dissemination................................................................................................................................ 31Exhibitions..................................................................................................................................... 31Work-shops.................................................................................................................................... 31Papers and journal articles............................................................................................................. 32Project web site..............................................................................................................................32External evaluation........................................................................................................................ 33

External expert assessment...................................................................................................33IPR................................................................................................................................................. 33Sustainability of the project results short and long term............................................................... 33


Recommendations for teachers......................................................................................................34References......................................................................................................................................34Appendix A.................................................................................................................................... 35

Questionnaires...................................................................................................................... 35Romania................................................................................................................................35Hungary................................................................................................................................ 37Norway................................................................................................................................. 37

Appendix B.................................................................................................................................... 38


Introduction.

The Leonardo Da Vinci Project DO-IT : Disseminating Open and Innovative Tools and Services for VET in quality Assurance, reference 2009-1-NO1-LEO05-01046, has been implemented in the time period December 2009 through to December 2011.

The project consortium consisted of the following participants:

3. Sør-Trøndelag University College (HiST), Trondheim, Norway4. Petrus Major University, Tirgu Mures, Romania2. BOKIK, Miskolc, Hungary3. University of Huddersfield, Huddersfield, UK4. Quality Management Software (QM Soft), Oslo, Norway

The DO-IT proposal specifies the basic scope of the project and through this scope defines the evaluation strategy as well as the evaluation itself.

Main aims.Do-IT is aiming at disseminating and raising the awareness of a brand new integrated blended ABT learning environment offering flexible and sound pedagogical delivery of level specific manufacturing industry Quality Assurance (QA) production process training to VET schools and SME organizations in Romania, Hungary and Norway. Do-IT disseminates new methods for delivering in-company skills upgrading processes that reduces the costs related to competence and knowledge transfer, and enhances production competence and know-how transfer to VET schools. This includes: * Educate QA VET instructors as Activity Based Training learning environment advisers *

Disseminate an innovative transfer system for in-company QA training of personnel, delivered on a just-in-time basis without distance limitations by utilizing video and mobile, brand new, interactive student response systems on iPOD Touch.

Do-IT utilizes blended learning methods that mix and merge: * A pedagogical model that utilizes ABT to follow the production flow of an object* On-site training, * Self paced on-line education, * High quality instructional video delivery of learning material to institutes, SME and VET

schools, into one competence transfer model.

Objectives.The Do-IT actions include: * Dissemination of the new ABT training principles for effective transfer of competence in

companies, institutes and VET schools, * Dissemination and delivery of an instructor training program (e.g. good practises) targeting

design, development and implementation of the learning environment tools for QA trainers.* Delivery of flexible training courses to manufacturing industry companies for dissemination

of the learning environment, training methodologies, and sound inclusion of instructional video into skills upgrading.

* Seminars, conferences and exhibitions for dissemination, raising awareness, and transfer of


expert knowledge and good practises to stakeholders at national and European level.

The new learning environment and the ABT model close the traditional gap between VET training and the industrial production process workflow. The methods are generic, whereby they are applicable to European wide manufacturing industry sectors (fabrication industries, VET schools and SME’s). They facilitate a innovative solution for cost- and time effective transfer of industrial production process know-how and technology knowledge to SME at a just-in-time basis.

Partnership.The consortium has a strong industrial presence with strong ties to QA companies (QM), distance learning methods, blended learning, video technologies (HiST), and manufacturing industry (UPM, BOKIK and UoH) leading to identification of inadequacies in current training systems and standards for effective instructor training delivery. The partnership has complementary expertise and tasks. They have cooperated successfully previously. HiST and QM personnel were key partners in the Mecca project.

Outcomes.* Disseminating ABT learning environment to VET schools and companies that certify QA

personnel in Romania, Hungary and Norway.* Awareness rising and delivery of instructor training to QA trainers. * Delivery of training courses to QA specialists. * Reuse of training material and technologies from previous projects

Impact: Participating training institutes, VET schools and companies may utilize the new ABT learning environment to offer a broad range of specialized courses at a European level.

The project outcomes is based on a previous Leonardo Da Vinci Project, MECCA, which was developed in the time frame of October 2005 through to September 2007. This project aimed to validate new blended learning techniques where video recorded events was mixed with ABT training principles targeting the welding industry and VET schools in particular.An European dimension was implemented through the use of harmonized guidelines for education of European Welders developed by European Welding Federation (EWF).

Development and implementation of new European Directives and product standards are a part of the ongoing work to ensure free movement of goods and personnel within the European Union which are one of the key ideas behind the Lisbon Strategy.

The EU Directives have a dual purpose:1) Ensure free movement of goods and services2) Safeguard the public interest for minimum quality and safety of the goods and services

The manufacturing industries are faced with increased responsibilities and the use of CE marking of products strengthen the manufacturers responsibility of product reliability. On the other hand such requirements leads to increased need for education and training in the relevant areas and for all stakeholders in the industries affected by the CE marking. In the different parts of the Directives requirements for independent assessments by Notified Bodies of the Quality Assurance systems being implemented by the manufacturer.


Some of the most important EU Directives are listed in table 1.

Directive number

Name Abbreviation

87/404/EEC Simple Pressure Vessel Directive SPVD

97/23/EC Pressure Equipment Directive PED

99/36/EC Transportable Pressure Equipment Directive TPED

89/106/EEC Construction Product Directive CPD

01/16/EC Conventional Rail System Directive CRSD

96/48/EC High Speed Rail Directive HSRD

Table 1. Relevant Directives for the welding industry.

However, the Directives are not the only regulations that establish requirements for product quality and thus the same requirements for education and training for personnel. The European Product Standards (EPS) have been developed and implemented over a 10 year period and the result are a higher demand of qualified personnel with cross border knowledge within the VET system as well in the industry. (With cross border experience we mean that the personnel not only have a relevant background for welding itself, as an example, but also relevant education and training related to Quality Assurance and implementation of such technology in fabrication).

Some of the most important EPS, related to welding, are listed in table 2.

Directive Product standard Standard title

87/404/EEC(SPVD)

EN 286 Simple unfired pressure vessels designed to contain air or nitrogen

97/23/EC(PED)

EN 13445 EN 13480 EN 12952 EN 12953

Unfired Pressure VesselsMetallic Industrial PipingWater-Tube Boilers and Auxiliary InstallationsShell Boilers

99/36/EC(TPED)

EN 13530

EN 14025

Cryogenic Vessels – Large transportable vacuum insulated vesselsTanks for transport of dangerous goods

89/106/EEC(CPD)

EN 1090 Execution of steel and aluminium structures

01/16/EC(CRSD)96/48/EC(HSRD)

EN 15085 Welding of railway vehicles and components

Table 2. Some of the most important EPS, dealing with welding fabrication.


All these standards are directly or indirectly requiring the implementation of ISO 3834 which is an application standard under ISO 9000 set of QA standards. Implementation of ISO 9000 standards in a manufacturing company is consequently not sufficient in order to meet the requirements in the EU Directives and EPS. EWF has developed a set of guidelines for implementation of ISO 3834 in the manufacturing industry as well as a harmonized database which lists all manufacturing companies qualified and certified according this standard. The ISOP 3834 certificates are granted through a harmonized network of Authorised National Bodies for Company Certification, ANBCC. This system ensures a transparent understanding and implementation of the system itself.

The project outcomes are also based of one of the key developments in the Leonardo Da Vinci project EDUMECCA, which was developed within the timeframe of January 2009 through to December 2010, Contract 143545-2008-LLP-NO-KA3-KA3MP. This project aimed at developing an easy to use, yet flexible Student Response System (SRS) for Ipod touch, Iphone and PC. The system should provide the instructors and teachers with new methods and services with possibilities to provide on-the-fly questions and responses which were ideal for problem based inquires and discussions. This methodology should then nicely fit into the ABT methodology implemented through the MECCA project.

The key objective of DO-IT should the be a combination of the best from MECCA and EDUMECCA projects with the requirements given through the EPS and EU Directives. Through this combination the DO-IT project would address some of the key hot topics within the fabrication industry in Europe and target an increasing need for education, training and competence for all stakeholders in this industry segment.

Target Group for the project

Do-IT disseminate and raise the awareness to VET schools and in-company training organizations in Romania, Hungary and Norway. This means:* Educate VET QA instructors as Activity Based Training learning environment advisers * Disseminate an innovative transfer system for SME in-company training

Do-IT utilizes blended learning methods that mix and merge:* A pedagogical model that utilizes ABT to follow the production flow of an object* Onsite training, * Self paced on-line education, * High quality instructional video delivery of learning material to institutes, SME and VET

schools, into one competence transfer model.

In addition the dissemination is sectorial:* Do-IT disseminates and exploits activities targeting quality professionals, like quality

managers and quality auditors from enterprises, that are providing QA training according to standards like ISO 9000, 9001 as 9004, 19011 (auditing) and application in industry.


Definitions

Activity Based Training (ABT)

With ATB it is understood that the training follow the production activities according the production path of a predefined structure or product.The core idea behind ABT is that you should produce something. This something can be a variety of things, from sausages, to tourist activity to a physical product. This means that you can use the ABT theory as long as you are going to produce something, whatever this is. In this context we are using the word product for the phrase something.But the production of the product must consist of a production process. That means in order to produce something then this production can be sliced up in production steps so that the whole production consist of a number of steps and the production can be described as a production process or a production chain in order to reach the final target, namely the product.

The production process must be clearly defined so that each step in the process represents an added value to the process itself.

The production process.The production process will consist of a number of defined production steps, each adding a value to the product. These steps can be described with a scope, purpose and a time schedule. However the start point for any production is that it contains a order. An order can be defined as a set of documents clearly describing the product which shall be produced (or one document with a number of appendixes).

Quality Assurance

Quality Assurance is the systematic process of verifying or determining whether products or services meet or exceed (customer expectations) or specified requirements. Quality assurance is a process-driven approach with specific steps to help define and attain goals. This process considers design, development, production, and service.This means however that a quality assurance process do not necessarily give you a high quality product, it may just as well give you a “lousy” product, but the process that gives you that product is planning, executing and monitoring according a Quality Assurance process.

Quality Control

Quality Control is a set of activities to verify and document that the objectives of the Quality Assurance system is met.

Student Response System (SRS)

"Student Response Systems (SRS)" or "Personal Response Systems (PRS)," are hand-held devices that help teachers interact with students during lectures. Through interactive question-and-answer sessions, instructors can engage students in course material by providing instant visual feedback to


the class. This allows the instructor to collect individual responses from hundreds of students at once and gauge student comprehension. For this report and in this project the definition relates only to the software system initially developed through the EDUMECCA project for SRS activities by using IPOD/IPHONE as user input media.

ISO 9000

The ISO 9000 family of standards represents an international consensus on good quality management practices. It consists of standards and guidelines relating to quality management systems and related supporting standards.

ISO 9001:2008 is the standard that provides a set of standardized requirements for a quality management system, regardless of what the user organization does, its size, or whether it is in the private, or public sector. It is the only standard in the family against which organizations can be certified – although certification is not a compulsory requirement of the standard.

The other standards in the family cover specific aspects such as fundamentals and vocabulary, performance improvements, documentation, training, and financial and economic aspects.

(http://www.iso.org/iso/)

ISO 3834

For the manufacturing industries a set standards, ISO 3834, with appropriate guidelines have been developed. These guidelines are intended to be used for the following purposes:

* Providing interpretation of the requirements in the EN ISO 9000 series of standards, as a guideline for specification and establishment of the part of the quality system related to control of welding as a "Special Process".

* Providing guidelines to establish specifications and welding quality requirements where a quality system according to EN ISO 9001 and EN ISO 9003 is not involved

* Assessment of the welding quality requirements mentioned in a) and b) above.

ISO 19011

ISO 19011:2002 provides guidance on the principles of auditing, managing audit programmes, conducting quality management system audits and environmental management system audits, as well as guidance on the competence of quality and environmental management system auditors.

It is applicable to all organizations needing to conduct internal or external audits of quality and/or environmental management systems or to manage an audit programme.

The application of ISO 19011 to other types of audits is possible in principle provided that special consideration is paid to identifying the competence needed by the audit team members in such cases. (http://www.iso.org/iso/)

EN ISO/IEC 17021 : 2011 is new standard which replaces ISO 19011:2002. This alteration of reference standard has no influence on this project.

http://www.iso.org/iso/


Other production processes outside welding

Do-IT project are not focused on welding as a fabrication method as such, although the reference projects are based on welding. The reference course material and procedures are consequently targeting the welding fabrication industry.

But as the course at BOKIT in 2011 clearly addressed, the ABT methodology can be used in different educational setting as long as the education structure aims at producing something that can be regarded as a deliverable. This can be a product, a service or whatever.

ABT (and combined with SRS) has then a far wider scope than welding fabrication as such.

What do we mean by Quality Assurance ?

Quality Assurance is a process that shall be used for planning and verification whether the defined products or services meets the defined requirements. Quality assurance has in other words a process related view in order to define a number of process steps which can be followed and monitored in order to meet the defined quality and goals. The process can be a full process covering all stages from initial scope of work and design, down through all fabrication or production elements until it reach the product stage. Or it may also cover only a limited scope of the process.This means however that a quality assurance process do not necessarily give you a high quality product, it may just as well give you a “lousy” product, but the process that gives you that product has gone through a number of phases like; planning, executing and monitoring according a Quality Assurance plan.

Companies and institutions having a Quality Assurance System have the options of having a Certificate as a proof of the implemented system. Such a Certificate can either be accredited or not.

Personnel have the adequate education may receive a Certificate or Diploma as a proof of passing an examination. Such certificates or Diplomas may also be accredited or not.

A common misunderstanding is that a Quality Assurance Certificate, either accredited or not, is a proof of high quality of the product. A Quality Assurance Certificate is only a proof of having a quality assurance system. But a quality assurance system is never directly related to the product quality.In the same way, for personnel, a Quality Assurance Certificate or Diploma only tells that the person has an theoretical examination with a given result.


Background.

The ISO 9000 series of standards was first published in 1987. The British Standard BS 5750 was the frame for the standard.

Fig 1. Shows the development cycles for the ISO 9000 series of standards ( from BSI Group).

Since the implementation of the first standards in the late 1980s the number of certificates have exploded from around 450 000 certificates in year 2000 to more than 1.000.000 in 2009. ( China represents approximately ¼ of all these certificates). However the growth rate in number of new certificates has in the same time period dropped from 33% down to 8% (ISO Survey 2009)

One may say that there are a number of reasons for this explosive growth in number of certificates, and also for the drop in the growth rate:

1. The first market focus for implementing QA systems was the focus on competition. The implementation of a QA system and more specifically a ISO 900 system would give the company an image of Quality which would be an marketing advantage. The argument has been that implementation of the standards will increase the market share or give the company a possibility to increase the profit margin.

2. The second market focus was based of a number of studies telling that companies implementing a Quality Assurance system would give financial benefits (11) The ROA (Return On Assets) The conclusion is that the companies seeking certification has an improved performance measured as ROA.One of the underlaying questions are that if a company is well managed it is easy to implement an ISO 9000 system. However, if the management is not well organized and the company do implement an ISO 9000 system, then the company most probably continues to be bad managed, but now with an ISO 9000 system.

3. The implementation business around the ISO 9000, with accreditation, course development and so forth has been expanding over the years.Today this activity has become a huge business opportunity for a number of stakeholders in the industry, in the education sector and in general business. This business opportunity has lead to a wide variety of offers for certificates, diplomas and so forth with and without a real value (or a real substance).It is too often seen, in meeting rooms, in receptions and so forth, diplomas and certificates


with reference to ISO 9000 or other Quality Standards, but the practical value is zero.4. The costs of implementing and also maintaining the Certificates and Diplomas has grown.

The resources required for companies to maintain such system has also grown to a stage where many are reluctant to implementing it unless it is a contract requirement.

5. The relative decline in number of companies implementing ISO 9000 indicates that the market now seem mature for this Quality assurance standard. It also means that a number of topics do not fit well under the ISO 9000 umbrella. It also means that industry representatives have seen that the implementation of an ISO 9000 system has been too general in a number of cases so that the certificate do not meet minimum requirements. However new standard appears, like ISO 14001 Environmental Management System, ISO/TS 16949 Quality management System for Automotive production, ISO 3834 For Welding, ISO 13485 Medical Devices, ISO/IEC 27001 Information Technology, ISO 22000 Food Safety Management System

The standard ISO 3834 covers quality requirements for fusion welding. Welding is regarded as a special process in ISO 9000, which means that it is not possible to verify the product quality without destroying it. Therefore you must embed the quality in the product from the beginning of the production process, or actually already from the design phase. To ensure a proper quality and to optimize the production costs, the entire process must be controlled from the very start. ISO 3834 defines these requirements that the manufacturer must meet in order to use good practise in their welding fabrication.

A Quality Assurance implementation may often consist of the following topics:Plan Establish objectives, measurable targets and processes required to deliver the

desired results. Ensure that this plan is approved and supported by top management

Do Establish a roadmap for implementing the process including allocation resources for all activities required

Check Execute the required processes and monitor these against the planned targets and objectives

Act Verify that activities conforms according plan and create non-conformance activities when required. Apply and implement required changes with consent from top management.

Table 3. Topics which are essential in implementation of a QA system.


Role of the project participants in the consortium.

HIST

HiST is the project technical and administrative coordinator, and adds fundamental technical- and pedagogical expertise to the project: * Coordination of the project (administrative and technical).* Adaptation of new pedagogical methodologies that extend the MECCA developed Activity

Based Training methods.* Applications of state of the art video streaming, videoconferencing solutions, and new iPod

Touch student response systems, for just-in-time transfer of in-company training/education and train-the-trainer solutions.

* The integration of traditional educational models with new pedagogical methods adapted for visual communication v) Hosting and maintenance of video streaming servers offering access to vivid high-quality online training materials and digital learning resources on a “just-in-time” basis.

UOH

This group will lead the adaptation of Do-IT learning activity infrastructure for web-services and multi touch pressure sensitive hand held devices like iPod Touch, and will contribute to the project in the following aspects: * Provide multilingual interfaces in a new type of student response system on iPod Touch or

iPhone* To update and extend the MECCA know-how transfer methodologies for skill development

in the QA sector by analysing the evolution of the European job market over the last few years

* To adapt the MECCA project know-how transfer methodologies at the national level for the selected markets, i.e. Romania, Hungary and Norway

* Develop a limited amount of multimedia material that help rising the awareness* Pilot the prototypes of the adapted instructor courses in Romania, Hungary and Norway.

The experiences from these pilots will be used to tailor the final QA courses. The final courses will be offered to the market through the activities in WP 6.

UPM

The "Petru Maior" University in its capacity of a training organization, will have the following tasks: * To prepare the project with the promotor; to research and accumulate materials in the area; * To participate in the project development and organize 1-2 instructor training courses, and

afterwards utilize Activity Based training in QA training courses. UPM will also organise a national seminar and promote the project results at international conferences and meetings;


* To implement technologies and ped. methods from the Mecca project in Open Distance Learning Centre;

* To organise symposiums and conferences for the dissemination of project results at regional, national and European level;

* To help completing the intermediary report (after one year) and a final report (after two years) including the audit of the results.

BOKIK

BOKIK role in this project will be as a dissemination partner that work with the implementation of the innovative transfer system for SME in-company QA training of personnel. This includes:* Raising the awareness of the project and its final products and results,

organizing 1-2 QA instructor training courses and afterwards utilize Activity Based Training in one of our courses.

* BOKIK will disseminate this project through its communication channels. Website – www.bokik.hu, by monthly newsletters. Progress reports and development activity including the promotional material can be distributed in the BOKIK membership which include of 60 VET institutions from the North Hungarian region.

* BOKIK wants to test out and implement the new software solutions into competence transfer models, and the new pedagogical methods that utilize modern video solutions in combination with training methodologies where theoretical and practical training a delivered according to the production path.

QMSOFT

QM Soft will also contribute expertise in the field of quality assurance know-how and the relevant QA topics for the project. QM leads WP 4 (Adaptation of Educational Material…) and WP 5 (QA) Quality and Evaluation Plan. * QM will as part of WP 4 teach a number of QA courses to specialists in cooperation with

VET schools in Norway. It is expected that this include a combination of on-site training, distance learning solutions, video streaming and video conferencing.

* It will together with HiST organize the recording of a limited amount of QA related video material for targeted instruction purposes. Furthermore, QM participates in the user requirement specifications, the dissemination of project results and the exploitation plan.

* QM will participate in the 3 national seminars, and be a key partner for dissemination of ABT to VET schools in Romania, Hungary and Norway. QM disseminates project results to major international conferences in Europe, as well as to industrial policy makers.


Project elements

Activity Based Training

The core idea behind ABT, which was developed through the Leonardo da Vinci project MECCA, is that you should produce something . This something can be a variety of things, from sausages, to tourist activity to a physical product. This means that you can use the ABT theory as long as you are going to produce something, whatever this is. In this context we are using the word product for the phrase something.

But the production of the product must consist of a production process. That means in order to produce something then this production can be sliced up in production steps so that the whole production consist of a number of steps and the production can be described as a production process or a production chain in order to reach the final target, namely the product.

The production process must be clearly defined so that each step in the process represents an added value to the process itself. The production process will consist of a number of defined production steps, each adding a value to the product. These steps can be described with a scope, purpose and a time schedule.

However the start point for any production is that it contains a order.An order can be defined as a set of documents clearly describing the product which shall be produced (or one document with a number of appendixes). This means at least the following documents should be present in the order:

1. Design drawings (if relevant)2. Work drawings (if relevant)3. Welding drawings (if relevant)4. Assembly drawings (if relevant)5. Functional description of the product ( mandatory)6. Delivery requirements including requirement for documentation (mandatory)7. List of materials (if relevant)8. Quality Requirements (including reference to relevant standards) (mandatory)9. Delivery schedule for the product or services (mandatory)10. Personnel requirements (requirements for knowledge and competence, certification and

so forth) (mandatory)11. Health, Environment and Safety requirements (mandatory)12. Requirement for Control documentation and traceability (mandatory)13. Relevant control reports (mandatory)14. Non-conformance report (mandatory)15. Corrective action Report (mandatory)

If the product is a service then item 1 through to 4 is not relevant, but then a use case, use scenario or detailed functional description may serve the same purpose.

Through the scope and purpose, other activities can be defined in detail for each production step. The teacher may also add information in detail for these steps or, the students may have as a special task, to describe the process in detail.

What is very important here at the order phase, is that the quality assurance aspect and routines


becomes an essential part of the production process as such. All activities in the process flow shall then be closely followed up through the quality assurance process.

Collaborative learning between the students.

The core idea is that understanding and knowing are mediated by objects created and shared by the community (Van Aalst, 2009). Within this context, learning takes place through a collaborative effort to create shared knowledge objects (Paavola & Hakkarainen, 2005). To be able to actively participate in collaborative creation of knowledge objects, students are expected to take control of their learning, to go beyond individual efforts, and to engage in productive collaboration with peers. The ida of collaboration is central and a key element in ABT. Apart from the pedagogical benefits, collaborative learning will stimulate the ability to cooperate in order to full-fill the activities that is a prat of the production process.

From an industrial point of view the collaboration efforts an ability may in many cases be more important than the results itself. The industry is rapidly moving into self-contained working groups with a high degree of autonomy within the framework that exists. Without a high degree of collaboration within and between production groups the administrative burdens may be very high. In a global production environment this is even more important.

The Student Response System (SRS)

The project outcomes are also based of one of the key developments in the Leonardo Da Vinci project EDUMECCA, which was developed within the timeframe of January 2009 through to December 2010, Contract 143545-2008-LLP-NO-KA3-KA3MP. This project aimed at developing an easy to use, yet flexible Student Response System (SRS) for Ipod touch, Iphone and PC. The system should provide the instructors and teachers with new methods and services with possibilities to provide on-the-fly questions and responses which were ideal for problem based inquires and discussions.

Key elements of the SRS system.

The SRS system consists of several elements.

1. A hardware component.The hardware component for the user is the handheld devices itself. This can be a multitouch device delivered from any supplier as long as it has a large touch screen and a WI-FI interface or access to normal telephone connection. A PC may also be used for the user interface.The other component is the server, which is operated by the system administrator which also might be the teacher.

2. A Software componentThe software is developed using the Representational State Transfer (REST) (Roy Fielding) architecture and using Web standards like:HTTP , URL , XML/HTML/GIF/JPEG/etc

This means that the overall architecture starts with the system needs as a whole, without any constraints, and then gradually identifies and applies constraints to the design.

The current version of the software do not use any database system for data storage. For further information about the hardware and software components, please consult the User Requirements Report


3. A User component.The User component again consists of different elements or topics:a) How to physically operate the system, form a students perspectiveb) How to physically operate the system, from a teacher perspectivec) How to implement the use of the system in the daily education and training activities

Quality Assurance

In this project Quality Assurance activities should be an additional dimension to the educational tasks and work that had been carried out through the MECCA and EDUMECCA projects.Do-IT is aiming at disseminating and raising the awareness of a brand new integrated blended ABT learning environment offering flexible and sound pedagogical delivery of level specific manufacturing industry Quality Assurance (QA) production process training to VET schools and SME organizations in Romania, Hungary and Norway.Other perspectives were the European dimension of Do-IT. HiST and UPM may disseminate the project results through their membership in the European Distance Education Network that includes most countries in Europe. Extension of new harmonised guidelines in combination with a educational framework targeting train-the-trainer, stimulate the cooperation and exchange of trainers between institutes, VET schools and industry. Thus, the Do-IT dissemination and exploitation activities should enhance the Lisbon strategy for free movement of personnel due to the harmonised educational framework for skills upgrading processes targeting QA personnel in industry.The work plan would disseminate the MECCA project results in training of QA personnel in Romania, Hungary and Norway. This includes user requirement adaptation for national markets, provision of instructor training courses in each of the participating countries, and validation of the new learning environment by arranging a number of QA specialists courses in the three countries.The focus on mechanical industry and VET schools are strong with dissemination relevance for umbrella organizations like EWF and others.

The evaluation process

Evaluation is an essential part of the project work because the project contains a number of innovative elements which have never been mixed before in one project. These innovative elements have never been implemented in VET organizations targeting welding activities in Europe as well. Neither have umbrella organizations like the European Welding Federation (EWF) approached the innovative elements from his project into their current work for harmonized guidelines related to education of welding personnel at different levels.

Some of the important elements of this evaluation process will be:

• Ensure that the continuous work can be reviewed for corrections and quality improvements• The evaluation may give valuable feedback to the course providers in order to have a

continuous improvement and development of their courses• The evaluation will tell the participants if they are approaching the project objectives or not

and allow the participants to carry out corrective actions


• The evaluation will also inform the stakeholders in the process , like managers, participants, VET supervisors, industry participants and so forth, if the project participants are moving in the right directions or not

Objectives of this document

The objective of this document is to give feedback on the deliverables and work done as well of work in progress where data are available.The feedback is divided into different elements like:

• Has the project objectives been met ?• If the objectives not have been met, then what has been the alternatives ?• If alternatives have been defined, how has these been met ?

The document covers first an overall review of the project and its objectives. This is done in order to evaluate if the objectives of the project actually are covered through the actions carried out in the project itself.Then it will evaluate the the training courses itself and training courses which have been carried out with target groups in:

• Romania• Hungary • Norway

At the end of the documents, findings from external expert has been implemented together with the questionnaires used for evaluation.

Instructors, teachers and students

The profile of the different groups participating in this project may be quite different depending on their role and their tasks and activities in the project. They may also have different interest in the project, creating different biases to the content development and the delivery structure of the content.

Group Profile Teaching technology

Instruction at University

level

This will be personnel at minimum master levelCompetence profile: Master level or higher. Very often with high theoretical background and relative little or limited experience from industry. However very often with a broad research background.

All modern technologies available. A variety of technologies are used, but often depending on the lecturer.

Instructors, teachers in

industry

This will be personnel working in professional training centres either organized as a training institute or as a training branch in an industrial company. Competence profile: Welding specialist or engineer

Traditional face-to-face technology with use of CD/DVD material in order to highlight technical matters.


with special education or experience in production.Usually this personnel have daily contact with fabrication facilities and fabrication technologyAge: 30 - 45 years, male

Training: Well equipped training facilities

Instructors, teachersin VET

This will be personnel working in the school system covering both traditional education and also life long learning with adult personnel for trainingCompetence profile: Specialist with documented competence from fabricationAge: 40 – 60 years, male

Traditional face-to-face technology with use of CD/DVD material in order to highlight technical matters.Most of the personnel have additional education related to pedagogic.Training: Lack of equipment in the training facilities

Students in industry

A mixture of male and female students in the age group of 25 – 35 years.Little or no background from industry or from other educational areas. Weak background from industry

Reason for seeking education and training:-promotion-seeking a job

Students in VET

Most students are male in the age group of 18 – 30 years. Background will be from the school system or from public supported educational programmes.

Reason for seeking education and training:-get an education-being promised a job

The user market

The European welding fabrication industry has a turnover of more than 1.6 billion Euros per year and offers free movement of products, services, personnel, etc. Subcontracting of knowledge-based production is extensively used due to short manufacturing schedules.

Table 4. Data from DVS in Germany tells us that in selected European countries more than 1.1 million people are directly employed by welding activities. This means if we add up the rest of the


EU countries then that number of employees will be close to 2 millions.

These people represents a population where the education has been targeted one item or trade only. The cross-profession education has seldom been offered to the people in this population. The DO-IT project ideas can then be seen as a possible route for life long learning in this case.

Stakeholders in the market.

The stakeholders in the market originally targeted by the project is as follows:

• Instructors in VET schools• Teachers in VET schools• Fabrication companies in need of qualified personnel at different levels• Organizations for welding fabricators and employees, like EWF• Educational institutions educating personnel at BSc and MSc level for mechanical industry • In-company training organizations• National training organizations

Methods employed for internal and external evaluation

Internal evaluation has been carried out through interviews, reading reference material, questionnaires and other project report which has been submitted within and up to November 9th 2011.

External evaluation

External evaluation has been carried out by an external evaluator who are familiar with the ABT methodology and also the SRS system.External evaluation has been carried out by interviewing some of the participants in combination with a study of material published through the project web side.

Analysis and discussions

Evaluation

The evaluation methods have differed slightly in the different countries. For most countries the evaluation has been carried out using questionnaires. In some cases additional complementary interviews have been carried out. However as we have seen some of the courses have been different and targeting different user groups. This makes the direct comparison more difficult.The background in education, training and competence for the participants in the work-shops and training courses covered a wide range of educational levels. The use of different computer technologies were also varying, creating difficulties in direct comparison of the observations.

The SRS training courses have however been quite uniform in all countries and the findings gives a


good picture of the impact such systems may have in these environments.

Project objectives versus real activities

The project objectives aimed at disseminate an innovative transfer system for in-company QA training of personnel, delivered on a just-in-time basis without distance limitations by utilizing video and mobile, brand new, interactive student response systems on iPOD Touch.

Due to the differences in the national markets as well as local support from VET Schools and industry combined with the local industry structure, the project objective could not be met in the integrated manner that were foreseen.

However the national participants had to adjust to local requirements and the need of the stakeholders in the local environment. The project adapted the objectives to this reality and the consequences were that the courses had to reflect the local needs which raised after the first meetings with the stakeholders.

User Requirements Report.

The User Requirements Reports consists of three chapters. These chapters go into the national details and requirements for each topic. And it gives an overall summary and conclusion for each chapter as well;

1. Training requirements of the manufacturing Industry in Quality AssuranceConclusion for this chapter:Three occupations are identified in quality assurance: Quality Managers, Specialists in Quality Systems, Quality Auditors. Proper curricula are established for VET in these occupations. Also demands for a quality VET program are established.

2. Technical Specifications, equipment solutionsConclusion for this chapter:Equipments recommended in quality assurance training are:- for face to face training: smart-board, camera documents, projectors,- for distance training: videoconferencing equipment,- evaluation of courses: ipod, iphone, SRS system.

3. Pedagogical methodologiesConclusion for this chapter:Pedagogical methodologies we recommend for training in quality assurance areactivity based training for teaching in the occupations of quality managers, qualityprofessionals, quality auditors. For training at distance the videoconferencing isrecommended. For training evaluation we recommend to use the SRS system.

Project fragmentation

The municipal of Miskolc was originally one of the main industrial areas in Hungary. However after the collapse of the socialist government and a following privatization of the industry followed


later by the financial crisis, this region was very hard hit by the new economical realities, wiping out a lot of the traditional industries.The new industrial environment consists mainly of SME enterprises and service oriented activities

Targu Mures region have more than 8500 SMEs. These are largely centred around chemical-, food-, wood- and textile industry. However Targu Mures is a centre for general and Higher Education.

The target industry for two of the three participants were missing and the project had to be fragmented in order to meet the requirements of the local stakeholders. This is also reflected in the User Requirements Report where the conclusions are not relevant for all participants.

The training courses.

The DO-IT project addressed Quality Assurance in industry and specifically welding industry. In the welding industry sector ISO 9000 is used as a general management system, but another application related and complementary standard is used, namely IS0 3834 towards the fabrication itself as discussed earlier in this document.Referring to PED and other EU directives as well as EPS related to mechanical industry and production, then that standard (ISO 3834) is the important one, for internal use in companies. However, one will then assume that an ISO 9000 standard has been implemented as an overall business standard for the company when also ISO 3834 is implemented.

This difference in QA standards has not been discussed specifically in the project, nor in the User Requirements Report in either.

UPM and BOKIK do not have relevant welding fabrication industries as their cooperation partners in the project which could utilize the MECCA courses directly, nor do they have relevant education and training courses which conforms to the EWF requirements for harmonized education and training for the welding industry.On the other hand both UPM and BOKIK has contacts with VET Schools of different kind and other educational suppliers as well.

The User Requirement Report (D 3) has therefore focused on ISO 9000 as the QA system and the report concludes with identification of three level of QA competence, namely Quality Manager, Specialists in Quality Systems and Quality Auditors.This identification is correct when it comes to implementation of ISO 9000 and also to verify such system using ISO 19011.

The DO-IT project should be based on the MECCA and EDUMECCA results and ISO 9000 are not the target QA standard for the industry segment approached in these projects and for the educational courses developed as well.

For the DO-IT project the consequences are that the courses delivered in Romania and Hungary will have a national focus that will be different from the focus in Norway where mechanical industry and courses are in line with the MECCA and EDUMECCA results.

These differences one have to have in mind when one look at the evaluation at a later stage in this


report.

Implementation of SRS has been carried out as a free-standing tool for education and training as such. No direct coordination of SRS possibilities for exploitation of SRS in the ABT courses have been done.

What is lacking however is a set of courses which are developed according the ABT principle, adding a QA component and utilizing the SRS system at the same time. Such an integrated course structure would should be the target for the last 6 month of the project.

Instructors, teachers and students

It is interesting to evaluate the background education and competence of the teachers and instructors participating at the courses in order to see if there are some similarities between the countries involved in the project or not. If similarities occur then the dissemination of the project results might be rather straight forward and experience from one country could easily be transferred.One might also expect that the expertise and competence would be dynamic through the project period.The same conditions are valid for the students. Their background, education and competence would be of interest when we evaluate the possibilities for future dissemination of the project results.

Instructors and teachers.

In this context there are differences in the term Instructor and Teacher.A Teacher is defined as a person that is more focused on the theoretical education and training. A teacher in this context do not involve her/him-selves in practical skills transfer or skills upgrade.An Instructor may be defined as a Teacher with additional practical knowledge and competence to also cover skills transfer and skills upgrade. But--- an Instructor may also not cover all Teachers aspects but mainly focus on skills upgrade and skills transfer.

Due to the fragmentation of the project, the main focus for training has been the Teacher aspect. Little, if any focus has been on the Instructor side.

Students.

The students may consist of a wide variety of personnel, both related to their theoretical background, their age and their focus on higher education at University and University College level versus VET school level.

Training and evaluation in Romania

The Quality Assurance courses referred to in Romania covers ISO 9000 topics and more specifically ISO 19011. The target user groups were Quality Managers, Specialists in Quality Systems and Quality Auditors.This is not directly related to mechanical industry and ISO 3834 has not been a topic due to industry


background of the companies participating in the courses.

However the Quality Assurance course targeted the Quality Audit process was addressed.

The system flow is sown in the following table. Of importance is that this course is the ONLY course in this project where practical education has been integrated with the use of the SRS system as such.

Results of the questionnaires have been published on the project web site.

The ABT methodology for quality management system audit course according to ISO 19011 and ISO 9001 Evaluation of theoretical achievements with SRS

Module no Quality audit process - ABT activities

Verification and control process - ABT activities

Type of module

SRS evaluation

1. Introduction in the QA course

Introduction to the QA course, scope of the education in quality

Organization Theoretical 13 questions

2. Contracting the audit

Delivery of contractual documents including domain and criteria of audit

Contract reviewEstablish the criteria for audit

Practical -

3. Establishing the audit team

Nomination of chief auditor, selection of auditors

Qualification of auditors, Criteria for chief auditor

Theoretical 11 questions

4. Analysis of quality management system documentation

Analysis of ● Quality manual, ● System procedures, ● Operational procedures

Degree of fulfilment of audit criteria

Practical -

5.Audit planning and execution

Knowledge of audit documents, main steps for planning and execution

Degree of organisational comprising in the audit process

Theoretical 15 questions

6. Elaboration of documents for audit

Creating the:● Audit plan, ● Audit questioner, ● Nonconformities

Control of documents format according to ISO 9001, ISO 19011 and

Practical -


report,● Process sheet● Audit report

system documentation

7. Audit planning

Elaboration and delivery of audit plan – agreements between auditors and audited organization

Control of audit plan according to organisational chart and system documentation

Practical -

8. Audit execution

Examination of organization, filling ● Audit questioner, ● Nonconformities report

Control and verification before, during and after audit – control of proofs supporting nonconformities

Practical -

9.Audit report and follow up corrections

Knowledge of audit reporting activities and implementation of corrections

Report mechanism Theoretical 14 questions

10. Audit report elaboration

Filling the audit report and delivery to the audited organisation

Control of conclusions in the audit report

Practical -

11. CorrectionsEstablish corrections, corrective actions

Evaluation of envisioned efficiency of corrective actions

Practical -

12. Follow corrections and corrective actions

Evaluation of corrections and corrective actions

Control of documents supporting corrections and corrective actions. Evaluation of efficiency and efficacy of proposed corrective actions

Practical -

13. Appreciation of the Quality Audit ABT course and SRS evaluation

Participants list at the courseEvaluation questioner

Evaluation


Training and evaluation in Hungary

Two separate set of courses have been held in Miskolc

1. One set of SRS courses conducted by Hist. No Reports or conclusions have been submitted by Hist from these courses.

2. One set of courses targeting QA and how to create ABT courses.BOKIT has created a course summary and translated the course material to Hungarian.

The following conclusions are based on the BOKIT report.

Number of participants were 20 people. From these 20, 5 represented educational institutions, 10 represented the private/ industrial sector and the remaining ticked off other category.15 persons responded in the questionnaire.

Question 1: How do you evaluate the presentation ?

Presentation was very useful and instructive and interesting : 11This methodology is unusual and new: 3Presentation was ? : 1

Question 2: Do you use similar process at Your institute ? If Yes, what kind of ?

NO: 13Use a QA system: 2

Why do they not use it ?

This system needs to be harmony between industry and education system, as well as the necessary devices are needed to be available. In Hungary it has to be developed.

Question 3: Do you think you can use this training in Your lectures ?

Maybe: 9Yes : 3No; need more information: 1

Question 4: Do you think that ABT training has value adding at your work ?Yes : 12Need to change the approach to use it: 3

General comments and remarks:

• This methodology is excellent in those institutions in which there are industrial practical


studies, because the students know that they need the knowledge which they have to learn.

• This methodology can be used in master education or engineer education.

• The change of approaches in Hungary is very necessary to this methodology can be used.

• This system can be used efficiently, helps the practical studies and especially teach the students to be responsible

• The method should be tested in many education institutions of Borsod-Abauj-Zemplen county.

Training and evaluation in Norway

Training in Norway has been carried out as two distinct different type of courses.1. Courses related to SRS and the use of the SRS system as such2. Course related to ABT and Quality Assurance targeting VET Schools and Industry.

The feedback from Hist on the use of SRS system is given in Appendix B

A central part missing however is an evaluation of the SRS system as such compared with other similar response system on the market.

Results and conclusions from the ABT and Quality Assurance course held in Norway. One course has been held with 7 participants.Topic: ABT and Quality Assurance in welding

1. General comment to item 1. Quality Assurance in Welding.After a presentation of the ISO 3834 requirements and presentation a set of procedures developed in the DO-it project as practical examples, it turned out that a number of these procedures were not implemented in the company.We can mention:* Procedure for evaluation of offers. There were no evaluation of the offers sent to 3-party from a welding point of view* Procedure for contract review was missing. There were no contract review from a welding point of view and the possibility to review the design and work drawings were not a topic that has been evaluated. A contract review could be an opportunity to evaluate the contract in order to enhance its design so that the equipment and the production facilities could be utilized with maximum effect.* The engineering manager and the purchase manager were not present at the presentation and it was decided that these people shall have a separate internal meeting in order to plan the development and implementation of adequate routines for such procedures.

2. Routines for maintenance and calibration of equipment were missing. Maintenance and calibration were carried out but not according a quality plan and quality schedule. In order to meet requirements in some of the new offshore contracts were traceability of the welding parameters might arise, then relevant procedures should be implemented.

3. Routines for education and training of welding personnel.Relevant routines for education and training were missing.


The company had hired a number of welders from Poland. However not all of the welders could communicate in the English language although they could understand what was said. Although the welders had relevant background and certificates, it was observed through detailed discussion around the practical work, that the understanding of the role of the welder and the responsibility of the welder were not in line with the management.Running through a training example with ABT training followed by an extensive discussion, it turned out that the hired welders did not understand their responsibilities in the production in the same way as the management defined it.A major clarification effort was done through use of DO-it procedures and examples from the ABT course, but the time was too short to go into all aspects and details of this topic.* Actions should be taken to create an educational program for the welders in order to highlight the responsibilities of all welding personnel and how this responsibility would influence their behaviour in production.* It also turned out that the basic skills in reading and understanding work drawings were missing. The knowledge of the different welding symbols and how they were used in a work drawing did not exists.* Actions should be taken to create a training course targeting skills upgrade of the welders related to the understanding of work drawings with welding symbols. Such a course could be offered by Vitec or the company could train the welders themselves.* Other topics related to welding personnel.HMS was discussed according the relevant part of the standard and routines have been implemented according the standard. However relevant training regarding possible breach of threshold values and how to report this was not dealt with.

4. Item 2 Quality Assurance as a process.The Quality Assurance system has not been implemented as a process related topic for the welding fabrication. That means that it do not cover the use of sub-suppliers in a correct manner. The same is other suppliers not directly related to production itself. More about this will be found under the topic documentation.

5. Item 4. Documentation and traceabilityThe company do have a manual system for documentation and traceability. However during the presentation and later through the ABT course, it is seen that not all participants in the fabrication process have the same understanding of what documentation and traceability really means. Actions must be taken in order to ensure that traceability and documentation can full-fill the offshore requirements which will be implemented in the next orders.Of special importance will be the actions on non-conformances and closing of non-conformance reports.* Reporting routines, procedures and documents must be designed for this purpose. The examples from the Do-it project may be used as reference and examples.* Responsibilities for maintaining documentation and traceability must be defined.

6. Item 5. IQSIM welding simulator was demonstrated during the course in order to highlight some basic process relations.* To stress the importance of fit-up and welding production. That means the preparation work and its consequences for future work down the line* To stress the relation between material quality, defined through material certificates and the welding processes. This may influence the purchase policy* The relation between repair rate and repair costs. This will highlight the importance of doing things right from the beginning and also the welders responsibility in the fabrication process.

7. Item 7. ABTExamples on how to implement ABT in their own education with examples from production


orders was shown. The company needs to implement a set of educational modules in order to reach a fully operational quality assurance system. By using ABT in this process the education and training will be highly relevant for all parties involved.After a short description and demo of SRS it was found that with the low number of participants in the courses, 4 to 6 participants SRS would not be a relevant tool to implement for the bespoke training courses.

8. Evaluation of the presentation.The presentation was evaluated verbally and the result was noted on a standard evaluation sheet.

9. Conclusion.Additional training of bespoke nature has to be implemented. ABT is ideal for developing such targeted courses using clients data as the basis for the courses itself.

Dissemination

The key questions in a dissemination plan is: what, to whom how and hen to disseminate.A structured dissemination plan should be a key activity in any projects. A strategy with relevant stakeholders, realistic objectives and a time schedule should have been presented.A sound dissemination plan should have included some of the following elements:

1. What is the expected results ?2. What kind of needs do the project respond to ?3. Types of dissemination activities ?4. Who are the target groups ?5. What are the most appropriate channels ?6. What is the best time schedule ?7. Will the required promotion material be available for these activities ?8. Which resources shall be used from each of the participants ?9. Shall other resources be allocated for the dissemination ?10. How shall the project be branded, i.e. what type of profile do we want the project to have ?11. Shall social media be used in the dissemination ?

Exhibitions

The project results have been exhibited with success at:

Industridagene, Lillestrøm, Norway September 2010Tekniske Messen, Stockholm, Sweden October 2010ON-LIne Educa, Berlin, Germany December 2010

A high number of attendees visited the exhibition and got first class information about the project. A number of demos were carried out by the exhibition staff.

More information can be found on the project web site.

Work-shops

A number of work-shops have been held at different locations and with different thematic focus.

Trondheim, Norway May 2011Lake Districk, Uk June 2011


More information can be found on the project web site.

Papers and journal articles

Presentation:Innovative Tools and Models for eLearning in Vocational Education and Training, by Prof. Liviu Moldovan at the conference eLearning Africa 2010, 5th International Conference on ICT for Development, education and Training, Mulungushi International Conference Centre, Lusaka, Zambia, May 26 – 28, 2010

John B. Stav, Erik Engh and Ketil Arnesen, Collaborative Workplace Learning by use of Activity Based Training Methods in Manufacturing Industry, to be published in the proceedings from the International Intend conference, 7-9 March 2011, Valencia, Spain

John B. Stav, Experience with Product Oriented Training and Mobile Learning in Education and Vocational Training, Proceedings from the International Conference Online Educa 2010, December 1-3, Berlin, 2010

John B. Stav and Erik Engh, Designing and Developing Product Oriented Training Methods in Vocational Education, proceedings from the European Conference on e-Learning 2010, 4-5 November 2010, Porto, Portugal

John B. Stav, Erik Engh and Ketil Arnesen, Experiences with product oriented training and mobile computing in education and training, Proceedings from the International EduLearn conference, 5-7 July, Barcelona, Spain

John B. Stav, Trond M. Thorseth, Kjetil L. Nielsen, Gabrielle Hansen-Nygård, Joan Lu and Pascal Pein, Experiences with Use of Open Web-Based Student Response Services in Sciences, proceedings from the SEFI seminar in Wismar, June 2010, Germany

John B. Stav, Kjetil L. Nielsen, Gabrielle Hansen-Nygård and Trond M. Thorseth, Experiences Obtained with Integration of Student Response Systems for iPod Touch and iPhone into e-learning environments, Electronic Journal of e-Learning, Volume 8, Issue 2 2010 (179-190)

For a complete list see the project web site.

Project web site

A dedicated project web site is a frequently used channel for dissemination of the project activities.For the Do-IT project the address is: http://histproject.no/node/32

The web site contains two parts, the public part with information and content that can be viewed by everybody and a members area where content for the project members are stored together with minutes, reports etc. This deviation of a public and internal part is common for projects like this and it works well.

However the current web site has been designed as a Hist project web site.The following items are usually implemented for International cooperation projects. It is not specified in the contract, however it is missing here. No project logo has been designed, Logo for the different participants are missing. Reference to partners web-sites are missing. Such references and items ought to be implemented.

http://histproject.no/node/32

http://issuu.com/academic-conferences.org/docs/ejel-volume8-issue2-article129/2?mode=a_p





Also logo from the Life Long Learning programme is missing on all pages (tested on Chrome and Firefox).

External evaluation

External assessment has been carried out by Mr Bela Gayer. He has extensive experience from education, industry and use of ABT for education of welding personnel.

External expert assessment

The report was Created by: Bela Gayer

Company: 3XGBt, Budapest, Hungary 1148 Ors Vezer tere 17

Conclusion of the report:

The consortium has been been trying to full-fill the outcomes and impacts of the project in their best manner. However the industrial reality have made it difficult to address all aspects of the project in the foreseen manner. Different technological background combined with industrial know-how and expertise have led to different solutions for full-filling the project requirements.

The resulting courses and activities are consequently more diverse than originally planned. The experience however is that the project ideas should be further disseminated on an European level to VET schools and companies as well as at higher education level.

IPR

Do-IT do not contain major new innovations or research activities. However new video material, course material and other content have been created throughout the project period. For future dissemination of the project results an Intellectual Property Rights (IPR) agreement is mandatory if later conflicts shall be avoided. An IPR agreement shall include all, if any, patents, copyright claims or any other category of intellectual property rights.It is, in the partner contracts stated that IPR regulations should be signed, but this has not been done. An IPR agreement will also be useful if any business plans are developing.Such a agreement should have been created at the start of the project itself.

Sustainability of the project results short and long term

Sustainability is defined as the capacity of the project to continue with its existence beyond its end. Sustainability of its results implies the use and exploitation of the project results on a long term basis.

It is difficult to see that the project has created any new sustainable results as such.However, when that is said, then the SRS system and the ABT methodology that has been used in this project seems to have a sustainable future in all the countries involved in this project. New knowledge and competence have been obtained by the project members and one may say that this


result is the most important factor for future sustainable use of the project findings.

However, the dissemination activities carried out in this project will lead to new European projects where SRS and ABT will be further developed and exploited in a wider geographical context. One might say that the valorisation factor of this project will be positive. Although not directly planned, it is foreseen that the projects outcome and products/ideas will have an European influence in the years to come.

Recommendations for teachers

The findings reported both from Romania and Hungary clearly states that both SRS and ABT are attractive technologies and pedagogical methods that will attract the students interest.As reported through numerous articles referred to on the project web site, the teacher -student interaction will increase and the unbiased feedback creates motivation and enthusiasm.

For the teachers it is however important to be aware of that the student generation are normally far more advance in the use of mobile devices than the teacher. When implementing the SRS system the teacher has to ensure that he/she can manage the use of the technology itself. But just as important as the use of the technology is the way the questions are asked in the teaching context,. Training, experience and discussions in order to obtain the maximum benefit for all parties involved will be crucial.

When implementing an ABT methodology one may also be aware of that few off-the-shelf courses exists on the market today. Some examples are published and also some free examples can be downloaded from different sites- The most complete examples resides in a LAMS repository. For more information see: http://lamsfoundation.org/index.htm

Implementing ABT will often require that additional learning content must be developed. The main reason is that most of the learning content available on the market in an off-the-shelf version are not process oriented. ABT will require a process oriented approach.

References

See the project web site for a complete list of publications; http://histproject.no/node/32

http://histproject.no/node/32

http://lamsfoundation.org/index.htm


Appendix A

Questionnaires

Romania


Hungary

Questionnaires are in Hungarian and resides at BOKIT

Norway

No questionnaires have been submitted


Appendix B

Evaluation rapport DO-IT.

1. Experience from use of Student Response System for modern mobile devices

1.1 What is a student response system?

Student response systems (SRS) have been used for many years, typically in large classes to increase the level of student’s engagement and learning. In literature SRS may have many different names, such as clickers, personal response systems, audience response systems, and classroom response systems. SRS are technology products designed to support communication and interactivity in classes. The technology allows an instructor to present a question or problem to the class, and receive answers from the students through a response device. A summary of all answers is presented to the teacher and the students to see. In other words, SRS is a communication system that allows the teacher to collect and analyze large amount of data and on behalf of these investigate whether learning has taken place. Research shows that’s such systems have the potential to facilitate several classroom processes such as; participation collaboration, physical activity, cognitive involvement, and self-assessment.A traditional SRS generally include a receiver for instructors, a collection of keypads (transmitters or “clickers”) for students and a dedicated software component. The software application is installed on the instructor’s computer such that the teacher may use it to create interactive presentations. Since the students use their keypads instead of raising hands to submit answers, the individual responses stay confidential from the rest of the students while result overviews are still available on the classroom screen by way of visual technology. There are several commercial systems available on the marked. They provide variations in functionality, and use infrared or radio frequencies to facilitate the methods for communication. Some systems are limited to multiple choice type questions, while other systems include yes or no, thru or false, as well as text and numeric responses. In Europe the price of commercial SRS constitutes one of the main factors that limit the penetration rate in education.

Figure 1. The online SRS Control interface and the mobile device developed in the KA3-ICT EduMecca project (1.1.2009 - 31.12.2010), contract no Contract 143545-2008-LLP-NO-KA3-KA3MP.


1.2 Active learning: Response-Communication-Interaction

Existing SRS systems based on clickers require that students receive a small hardware based device. For many educational institutions this become far too expensive, whereby it limits the utilization of new pedagogical methodologies based on anonymous feedback from students during training sessions. The new services uses the existing wireless network inside the institutions, whereby the educational institutions don’t need any more to invest in expensive equipment dedicated for voting sessions. The mobile computing based SRS system provides an economic or cost effective solution by utilizing widely available mobile, wireless multi touch pressure sensitive hand held devices, e.g. iPod Touch, iPhone or mobile devices, such that students may interact with the teacher through online questionnaires and voting systems. The teacher collects and visualizes the responses from class at the digital blackboard, by utilizing state of the art SRS decision process solutions. SRS mainly supports multiple-choice questions, but teachers can sample extensive data regarding their students' knowledge that is otherwise hard to obtain.The teacher gives the students a task, for instance a question or a problem. The students solve the task and respond by using the SRS either on their laptop or through their mobile handheld device, e.g. iPhone, as displayed in prototype SRS solution in figure 1. The results are displayed anonymously on the digital blackboard or by using a projector, whereby the teacher gets a knowledge map of the class. Finally, the teacher must decide how he/she will proceed. The results show if the class struggle with the current part of the curriculum, and he/she must decide the amount of time needed for that part based on the result. Thus, the SRS provides pedagogical methods that enhance interactive teaching models by enhancing communication and instructional feedback loops.

1.3 Choice of methods

There are many ways to use a response system, we have essentially separated between two methods, classic and peer instruction.

Figure 2. Use of SRS; Classical (one loop) versus Peer instruction (two loops). The question is posed on the blackboard. After the first or second loop the teacher explains why the correct alternative is correct, and why the incorrect alternatives are incorrect.


The classic methods works as follow:

* During the training session the students are asked a quiz question related to the topic being taught about.

* Students are given a few minutes to discuss the question and its alternatives in small groups (about 3-4 groups per group).

* They vote individually using an iPod* After the vote is closed, response distribution is shown to the students. The teacher

goes through the various alternatives, highlights the correct one and explains why it is correct and why the other alternatives then are less correct or incorrect.

The peer instruction method has the following steps:* During the training session the students are asked a quiz question related to the topic

being taught about.* Students are given one minute to work with the quiz question individually* They vote individually using an iPod* Students are then given a couple of minutes to discuss the quiz question in small

groups a few minutes (about 3-4 groups per group). * Before they vote again* After the second vote is closed, two response distributions are shown to the students.

The teacher goes through the various alternatives, highlights the correct one and explains why it is correct and why the other alternatives then are less correct or incorrect.

1.4 Evaluation results obtained when using the system

Graphs from evaluation of the student response system at HiST:

Students think it is fun to be at lectures where SRS is used


Use of SRS in class engage and activate many of the students

Use of SRS aid several students learning of the course curriculum.


Students grade the importance of peer learning as large.

Teachers explanation after the vote; which alternative is correct, incorrect, and why, is

perceived as very important for students learning experience.


According to the students, use of SRS has a great impact on their teaching:

* They get a chance to participate actively, which increases their involvement * Through discussing the quiz question with their peers, they get an opportunity to

engage in collaborative learning* They receive an immediate feedback on their learning* Through a thoroughly explanation from the teacher after the vote, they achieve a

greater understanding of the material being thought

2. Evaluation method

Following evaluation include results from four evaluations; one demonstration, one course and two workshops, all aimed at presentation, demonstration and use of SRS. In connection with these, numerous surveys, in form of questionnaire, have been conducted. The English word survey means overview. A survey is a standardized questioning of a sample of persons on any topic (Ringdal 2001). There are two ways to administer surveys. One is a written questionnaire where informants read the questions and indicate their response on a form. The other way is to use an interview format (Cozby 2003). In other words: survey research employs questionnaires and interviews to ask people to provide information about themselves and their experiences (Cozby 2003).

2.1 Questionnaires

There are a total of four questionnaires conducted in Trondheim, Norway and Lake District, England. Questionnaires in one of the most structured survey techniques (Ringdal 2001). With questionnaires, the questions are presented in written format and the informants write their answers. There are several positive features of using questionnaires. They are cost efficient and allow the informants to be completely anonymous as long as no identifying information is asked. However questionnaires require that the informants are able to read and understand the questions, and there may also be a problem with motivation; many people can find it boring to send time on reading and answering questions. Questionnaires can be administered in person to groups or individuals, through mail, on the internet and with other technologies (Cozby 2003). Current questionnaires were conducted using paper. After the demonstration, course or workshop was finished; it was handed out a questionnaire to the participants that they filled out manually.

3. Overview over completed surveys

3.1 Evaluation: demonstration, courses and workshops in use of Student Response System

Spring 2011: 06.04.2011.- Location: BOKIK, Hungary.- Selection: teachers, researchers, technical staff, administrator and decision makers. - Method: questionnaire- Number of informants: 14.

Spring 2011: 23.05.2011. - Location: Rica Nidelven Hotel, Trondheim.- Selection: Teachers, researchers and administrator from NTNU (Norwegian University of

Science and Technology), Trondheim. - Methods: questionnaire.


- Number of informants: 14.

Spring 2011: 23.05.2011- Location: Workshop HiST/PLU at NTNU, Trondheim. - Selection: teachers and researchers. - Methods: questionnaire.- Number of informants: 13.

Spring 2011: 01.06.2011- Location: Workshop at the CELMA Conference (International Conference on Education &

learning in Mobile Age), Lake District, England.- Selection: Teachers, research and students. - Methods: questionnaire.- Number of informants: 12

4. Evaluation results

4.1 Training course in use of student response system (SRS), BOKIK, Miskolc. 04/06/2011.

Number of participants: 14 participants (7 women and 7 men).Their positions ranged from teachers, researchers, students, technical staff, to administrator and others.The evaluation shows that the participants had a very good impression of the response system. The system was conceived as a simple tool to use, and was considered a positive element in relation to the ordinary teaching. Finances and lack of voting devices were highlighted as the major constraints for them in with regards to using the system. Most participants believed that they would be able to use the system, having attended the course.Additional comments were that the system should be developed further to accommodate exams and written tests, and that multi-lingual user manuals and interfaces would be desirable.

4.1.1 Graphs from questionnaire:

We have conducted several evaluations regarding use of student response system, following results appears in all of them.


4.2 Workshop at CELMA (International Conference on Education & Learning in Mobile Age).

Workshop title: Promote Active Learning with Student Response System for Modern Mobile Devices.


Number of participants: 12 participants (10 women and 2 men). Most of the participants were teachers, in addition to a few students and a researcher. The workshop included a short introduction and technical review of the response system, experiences and evaluation results from HiST; practical training; and a demonstration of the system with an emphasis on methodology and implementation. The evaluation clearly shows that the participants had a very good impression of the response system. The system was very easy to use, and was considered a very positive contribution in an ordinary teaching session. Some factors were identified as potential constraints with regards to implementing the SRS at the participants’ home institutes, such as internet connection; technical support; lack of devices and lack of familiarity. Despite these limitations, all the participants considered themselves qualified to use the system even after participating in the workshop.



4.3 Workshop in Trondheim, HIST / PLU, NTNU.

Workshop title: Promote Active Learning with Student Response System for Modern Mobile Devices.

Number of participants: 13 participants (9 men and 3 women), consisting teachers and researchers. The workshop included an introduction of the response system, a review of HiST’s experiences with the system; evaluation results; and practical training of the system. This was followed by an active demonstration targeted approach and local implementation of the system. The workshop concludes with an evaluation of both the workshop and the system itself. The workshop was considered very good. The participants reported a solid impression of the response system. The system seemed simple to use, and was definitely considered a positive contribution to ordinary teaching. Participants cited different stumbling blocks for a successful implementation at their respective home institutes, including financial issues, (lack of ) internet connection, (lack of) technical support, lack of devices, lack of familiarity with the system and their own personal confidence in using the system. The participants reported that they felt qualified to use the system after having attended the workshop.


4.4 Demonstration of the Student Response System for language teachers at NTNU, Western Chesterfield, Trondheim.

Number of participants: 14 participants (11 women and 3 men), mostly teachers and a few researchers. The demonstration was considered very good. Participants got a solid impression of the response system. The system seemed easy to use, and they perceived it as a positive contribution in a normal teaching situation. The participants reported some constraints with regards to their own usage of the system, highlighting finance issues, lack of internet connection, lack of technical support, lack of devices and familiarity of the system were put forward. Most participants considered themselves able to use the system after the demonstration.



4.5 Summary:

Overall, the evaluations show that participants have a positive impression of the response system. The system is perceived as a simple tool to use, and is considered a beneficial contribution compared to regular teaching styles. Some factors, however, was put forward as potential constraints in relation to participants' own potential use of the system. The factors that were most consistent throughout all evaluations were lack of voting devices, technical support and knowledge. This shows that although the participants have a positive impression of the system, there are some uncertainty factors. Our lesson from this is the importance of a thorough training, both technical and methodological, when SRS is to be introduced for, and used of new users. Make them confident in the technology and try to increase their knowledge about usage and methodology. The key word is in many ways security; to trust the technology, and at the same time possessing some ideas about a proper use of the system. Most of the participants believe that they are able to use the system after seeing it demonstrated only once. If their uncertainty factors fall into place, maybe there is a better chance for this to actually happen. When it comes to lack of voting units, the system is constructed so that it is available from various technological platforms. Whether students have these platforms available in their classes is another matter that each institution must consider.

ReferenceCozby, P. C. (2003). Methods in Behavioral Research, 8th ed. McGraw-Hill companies 2003. Ringdal, K. (2001). Unity and Diversity. Social Science Methodology and quantitative research. Fagbokforlaget Vigmostad & Bjørke AS 2001.

d 7 evaluation reporthistproject.no/sites/...report_from_qm_soft_final.pdf · do-it llp -project...

Documents