evaluation

EVALUATION

RIL PILOT PROGRAM

The Evaluation Questions

• The “learning gain competency” • Assessors understanding before starting, after

training and after application• Assessor views of usefulness• Usefulness of the complexity and volume of learning

in reaching decisions • Suggestions for improving


• The assessors training• Assessors confidence before training, after training

and after application • Improvements for future training• The discussion based assessment process (DBA)• Assessors expectation and experience of DBA • Each assessment tool - usefulness• Students capacity to prepare for the DBA• Students valuing of DBA


• Time / resource requirements• Time taken to complete the assessments• Constraints• Improvements • Overall• Effectiveness and validity of assessment • Confidence in process• Applicability of process

Data Gathering

• Two assessor surveys

• Site reports

• Survey/interview –19 candidates

• Discussion with each pilot group

• Some observation of DBA process

• Examination of documentation

Data Gathering

• Discussion with other stakeholders

• Employers

• RTOs

• Job network providers

• OTTE representatives

• Two industry training board representatives

What Assessors Said

• Education and training value of assessing and recognizing informal learning

• Assessment model can be effectively carried out – more so with modification

• DBA process works but needs further development

• Improved assessment questioning• Moderation to improve consistency• Better approach to volume of learning calculation

What Assessors Said

• The process tools were made to work but

• Adjust to reduce duplication• Better guide for candidate preparation and evidence

gathering

• Reduced paperwork

• Process can work for a range of candidates

• Results as valid as assessment within formal education and training

What Assessment Candidates Said

• Overall very positive• Most understood the process• Some expectation of credit into particular VET

courses• Mostly satisfied with discussion based assessment• Able to demonstrate what they had learned informally

• All satisfied with the result


• All found value from the process• Confirmed learning from a number of experiences • Evidence to support entry to further courses• Increased confidence to attempt further learning• Something to show possible employers• Many reported much greater awareness of of their

learning and personal value


• Candidates suggestions for improvement• More time to prepare for assessment• Clearer communication about evidence to collect

and relationship to further course entry• Reduced paperwork• Making language easier to read and understand

Other Stakeholders

• Employers• Four of 5 employers saw usefulness of the

assessment result and certificate to help decide on suitability for employment

• All employers could see value in assessing informal learning, but certificate language needs to indicate what candidates have done

Other Stakeholders

• Job network providers and RTOs• Some suggested employers would value as an indication of

employability skills

• Others said employers would only be interested in clear indication of industry competencies attained

• Still others suggested wording of the certificate to be more recognizable to employers

• Shared view that employers need to understand more about the RIL process assesses informal learning

Other Stakeholders

• OTTE and ITAB representatives• Confirmed that the RIL process remains

problematic – not assessment against industry competency standards

• However some value seen in• Possible indicator of fitness to enter VET courses without

credit or advanced standing

• Link to RPL RCC process – demonstrated capacity to identify the level to which candidates had operated

Conclusions and Recommendations

• Conclusions

• The RIL process worked to enable assessment against the learning gain unit

• Assessors and candidates value it• High degree of assessor confidence in results


• Conclusions (cont)

• Potential to assist individuals to better understand their own learning capacity and gain confidence for further learning

• RIL processes and tools need modification to improve understanding and efficiency

• Training future assessors needs greater emphasis on practice


• Recommendations• Greater emphasis on assisting people with clear disadvantages

that have deterred them from participating in further education and training

• That the VQA re-work the assessor package to improve understanding and efficiency

• Continue to explore possible links of RIL assessment results with VET course entry and RPL/RCC

• Examine extension of RIL assessment to employability skills making it more useful to employers and job seekers

evaluation

Documents