Bob SmartManaging Director

Warning Signs of E-BureaucracyIn E-Assessment / E-Portfolio’s

Providing E-Vocational Technology Since 1996

TM

What are the Dangers? Compliance for hundreds of Regulatory Bodies Common Accord & Code of practice conflicts (borders) etc Speed (requests for shortcuts by users . . The customer!) Savings Driven (requests on price . . Compromise!) IT Just because it can (Perception of what can actually be done v reality?) Trying to make it Equality Compliant (Too much can render it unfriendly) Everyone wants a different report (all centres need different hybrids) How many suppliers are there?

59 last count E-portfolio 271 E-Learner / testing Content Providers + . .?

How many Vocational Awards, and do they ‘appear’ the same on systems? Awarding Bodies, ALI, Ofsted, LSC, DfES etc etc Req bureaucracy (at mo) Open Source brings concerns for standardisation. Driven by Academic Research that ‘blanket’ e-portfolios without understanding vocational awards! (they don’t do them!) Web Based, Stand Alone, which is best? System Approvals, do colleges want 8 portfolio systems to cover A/B

approvals etc etc. Emperors Clothes!

What standards are already in place? Examples:

BS8788 The UK Lifelong Learning Profile (confused with NVQ’s) RDCEO Re-useable Definition of Competency Educational Objective. VDEX IMS Vocabulary Definition Exchange SCORM Sharable Content Object reference model CORDRA Content Object Repository Discovery and Registration / Resolution Architecture. Open Source . . . ! Etc. etc

Lets look at some examples that we have encountered! Different compliance and understanding of the word itself . .what is an e-portfolio? European verses English, Welsh and Scottish requirements.

Different standards and requirements exist even in our own borders! University interpretation of e-portoflio gathering steam based on repository model and academic

achievement. This could bully the market place at the influential levels (inadvertently) Open Source, fine for certain awards, but main stream vocational assessment it is not fit for purpose.

A/B need to approve systems, etc. Focus is on the learning and development element and not the practical and accreditation process

required to gain the award. Oh it’s a computer, it can therefore give me a report like this. . . That has no material benefit, but it

looks good! The actual level of competences required by the examiner / assessor (IT), is greater than is currently

in place and no specific development and training plan is available, only generic awareness of systems unless they purchase them!

What is an e-portfolio and should different regulations apply to different systems!

There are many different systems Repository of evidence (scrapbook of Qualifications, CV etc) Formal Accreditation (NVQ’s etc) (Either Assessor or Candidate

driven) Student lead, such as DIDA, OCR Working and Learning e-portfolio Possibly a private, working e-portfolio used as a staging post

before entering into another system.

So should different standards apply to different systems? Never mind Framework for achievement .

What’s the current thinking! Are the proposed action areas for aligning assessment feasible and appropriate? JISC feels that in this section of the strategy, clarity is needed on what is actually meant by e-

assessment.  Does it refer to “doing” online assessment (eg. a live online interaction) or does it refer to supporting assessment through e-learning systems.

Are the proposed action areas for assuring technical and quality standards feasible and appropriate?

The JISC believes that technical and quality standards need to be treated separately as the two are quite different.  JISC is only concerned with technical standards and does not feel that it appropriately constituted to tackle pedagogic or quality standards of electronic resources

But what is clear, is that no-one appears to have addressed the work of the electronic portfolio in vocational awards. It all appears to be academic based and focused on integrated e-learning. The terminology e-assessment has clouded the issues and as such creates a massive hole in the world of vocational assessment process / technology.

To sum up, unless we work together to demonstrate to show the route forward, then many years of work could be undone or at best misrepresented. To this end, someone must lead the way in creating a ‘standard’ for interoperability . . . E-skills? Catch up between developers being clever and deliverers trying to implement! (Gap getting bigger in some areas!, closer in others!)

Previous Speaker . . Thanks Paul, (Stakeholder Tools!) . . I expect the phone call then

What is the train of thought . . The mindset!

they’re not ready . . . which generation?

2004@1944

Examiners/Verifiers/SMT

Teachers/Lecturers/SOs/POs

NQTs/Graduates/Workforce of tomorrow

FE/Secondary student

Primary pupil

No innovation

Radical innovation

Paper-based

Computer-based

No innovation

Radical innovation

Paper-based

Computer-based

Bob SmartBob SmartManaging DirectorManaging Director

Providing E-Vocational Technology Since 1996

TM

Blake'sParliamentary Listed

What can we do?

The largest e-portoflio providers in the UK (as researched and indicated by C3 education), have joined forces and are to set up their own kite mark system, Jointly managed but administered by PaperFree Systems Ltd. These companies are:

PaperFree Systems Ltd FE penetration 41% ENVQ FE penetration 27% Quickstep FE penetration 21%

Creation of own Technical Standards with the assistance of the Awarding Bodies and DfES / LSC.

Development of the New E1 Assessor Award to train practitioners in best assessment practice.

Share common data infrastructure to allow interoperability with systems like Maytas and Cogisoft (already done) for Funding

Work with major training Providers to create common platform Already in Place with REMIT and Others.

Bob SmartBob SmartManaging DirectorManaging Director

Providing E-Vocational Technology Since 1996

TM

Blake'sParliamentary Listed