National College of Ireland

Certificate in Business Data Analytics, Level 8, 30ECTS, Minor award in association with Master of Science in Data Analytics

Table of Contents 1 Panel Report.............................................................................................................2 2 Declaration of Completeness of Response by Panel Chair......................................11 3 Programme Team Response...................................................................................12 4 Programme Schedule: Certificate in Business Analytics (30 ECTS).........................17 5 NCI Memo on Context of Validation........................................................................18

1 Panel Report

PANEL OF EXPERTS’ REPORT

1.1 PART I: PREAMBLE

1.1.1 Details of Validation Event PROVIDER National College of Ireland DATE OF VISIT 31st March 2014 PROGRAMME(S) EVALUATED

Certificate in Business Data Analytics

PROGRAMME (S) RECOMMENDED FOR

Certificate in Business Data Analytics

1

APPROVAL PANEL OF EXPERTS

Dr Dermot Douglas, Higher Education Consultant, Chair Mr Andrew Cobley, School of Computing, University of Dundee Ms Aoife D’Arcy, Director, The Analytics Store Dr Aisling O’Driscoll, Dept of Computing, Cork Institute of Technology Dr Bruno Voisin, Irish Centre for High End Computing, NUI Galway In attendance: Sinéad O’Sullivan, Director of Quality Assurance & Statistical Services, NCI (Rapporteur)

1.1.2 Summary The Expert Panel, having reviewed the documentation presented by NCI and considered the responses of the programme team during the course of the site visit; recommend approval of the following programme

Certificate in Business Analytics with six (6) conditions and eight (8) recommendations.

1.1.3 Context This validation process was undertaken with the agreement of QQI under its policy for the devolution of responsibility for validation sub processes which is outlined in Core Validation Policy & Criteria 2010, revised 2013 (p15). The devolution of responsibility for validation sub processes (DRSVP) at National College of Ireland applies to taught programmes up to level 9 of the National Framework of Qualifications that are developed and delivered solely by NCI within the jurisdiction of the Republic of Ireland. Programme disciplines are restricted to Business and Computing and programmes validated using this process are currently agreed with QQI on a case by case basis. The validation event proceeded without receipt of a desk review report from QQI. The panel members were nominated by NCI using criteria agreed with QQI. A representative of the Quality Assurance & Statistical Services (QASS) Office co-ordinated provision of submission documentation and the validation event. It provided administrative support to the panel during the

2

event as rapporteur. This representative was not a member of the panel nor did she take part in the deliberations of the panel. The QASS office acted as liaison between the panel and the programme team members in relation to the issuing of the panel’s report and the programme team’s response to that report. This report is the agreed report of the panel members.

3

1.2 PART II: REPORT OF PANEL OF EXPERTS

1.2.1 Introduction The National College of Ireland (NCI) has two schools – Business and Computing. Both schools provide a range of education programmes which are validated by QQI from level 5 to level 10 on the National Framework of Qualifications. The College has a diverse learner profile of approximately 3,500 learners. Of these, approximately 50% are part-time. The programme submission documents and the programme teams’ self-evaluation of the programmes were made available to the panel. A desk review report had not been received from QQI and was therefore not considered by the panel in its deliberations.

1.2.2 Examination of Programmes The panel met with staff of NCI involved in the design of the programme, to examine the programme submission against the criteria for the validation of programmes as stipulated by the QQI board. In this regard, the QQI’s Core Validation Policy and Criteria, 2010, revised 2013 was used by the Panel. The panel was presented with extensive and comprehensive documentation. There were some technical errors in the documentation which require correction.

1.2.3 Development and publication of explicit intended learning outcomes The programme submission documents, together with the outcome of discussions with NCI staff articulated the target learners’ prerequisite learning and any other relevant assumptions about programme participants. The programme was designed using QQI’s award standards for Science at level 9 of the National Framework of Qualifications. The panel was satisfied that the level of award sought was appropriate to the Learning Outcomes as describe, however . the goal of the programme which is outlined in section 2 of the programme submission document should be rewritten to better articulate the goal of this programme in its own right. The panel was alsosatisfied that the programme learning outcomes, supported by the module learning outcomes were consistent with the national standards of the award being sought.

1.2.4 Programme content, design and learning environment This programme is designed to be an ab-initio programme and to act as an exit award for learners who wish to leave the Postgraduate Diploma in

4

Science in Data Analytics or MSc in Data Analytics early. The Panel was satisfied that the programme’s content and learning environment are appropriate to the programmes intended learning outcomes. The programme team that the panel met is competent to enable learners to achieve the intended programme learning outcomes and to assess their achievements, in accordance with QQI’s Assessment and Standards 2009. The panel was advised by the President of the College that 2 full-time appointments were being made to the School of Computing to ensure appropriate resourcing for Masters research supervision and in the area of analytics specifically. The panel was satisfied that the College had satisfied itself that appropriate and sufficient human and physical resources were available to deliver the programme and that where gaps had been identified the College had committed to addressing them. Notwithstanding this, the panel expressed the view that the College must have sufficient teaching staff with an appropriate background in statistics and/or data science, data mining or machine learning in place prior the programme commencing. Members of the panel visited the College library which also houses Assistive Technology resources for learners, teaching spaces, and the Cloud Competency Centre and is satisfied that these resources are appropriate to these programme and learner population. Specific Modules Subject to the conditions and/or recommendations listed below, the panel was satisfied that the content of each module was consistent with the title of the module and supported achievement of the specific learning outcomes of each module. Statistics for Data Analytics The learning outcomes, content and assessment are considered appropriate. In addition, it is recommended that the learning outcomes for this module should refer to the application of techniques. The programme team may wish to consider reducing volume of subject matter covered in the course and adding an more appropriate introductory statistical text to the required reading. Data Warehousing & Business Intelligence The learning outcomes, content and assessment are considered appropriate. In addition, it is recommended that both texts by Inmon & Kimball should be on the required reading. The programme team may wish to consider a reweighting of the continuous assessment to 60% due to the amount of time spent on ETL. In-memory databases could be included here. Strategic ICT and eBusiness Implementation The learning outcomes, content and assessment for this module are considered appropriate. However, the panel considers that elements of law are missing from the programme and this may be the most appropriate module to include it in.

5

Programming for Data Analytics The learning outcomes, content and assessment are considered appropriate, however as outlined in 2.3.2 below learners should be guided when choosing this elective and made fully aware of the programming skills required to participate in it. Analytical CRM The learning outcomes, content and assessment are considered appropriate. In addition the panel recommends that an explicit section on web analytics should be included. Data Storage & Management The learning outcomes, content and assessment are considered appropriate. In addition, the panel recommends that graph and in-memory databases should be included. The panel also recommends that the programme team consider the addition of data visualisation to the programme content.

1.2.5 Enabling the achievement of the intended programme learning outcomes There was considerable discussion with the programme team regarding the admission criteria for learners and how it would be ensured that those coming from a non-technical background could participate successfully on the programme. The panel had a concern particularly with the Programming for Data Analytics module which is particularly challenging. The panel requires that learners are appropriately guided to ensure that the selection of this elective is undertaken only by those learners who have a sufficiently technical background. The panel also recommends that the programme team consider the user of a common platform e.g. R throughout all of the modules to ensure that those in particular, who are learning programming skills for the first time can reinforce their skills rather than being challenged further trying to learn multiple platforms.

1.2.6 Actions and procedures for access, transfer and progression for learners The Panel was satisfied that the procedures for access, transfer and progression are consistent with national policy. The programme submission documents would benefit from a re-articulation of the entry requirements – clearly denoting those for standard and non standard entrants. The panel was advised of a ‘bootcamp’ being available to learners who are admitted to the programme under non standard entry. The panel commends this initiative and would recommend that its structure and content is articulated further in the document and is not restricted to non-standard applicants only.

6

The panel is satisfied that the programme’s use of ECTS (credit) and provisions for Recognition of Prior Learning (RPL) is consistent with QQI’s Assessment and Standards 2009 and with relevant national policy including:

i. NQAI’s Principles and Operational Guidelines for the Implementation of a National Approach to Credit in Irish Higher Education and Training 2006

ii. NQAI’s Principles and Operational Guidelines for the Recognition of Prior Learning in Further and Higher Education and Training 2005

.

1.2.7 Education and training needs The panel is satisfied that these programmes satisfy a current need in the Irish education market as demonstrated by the rise in the number of programmes available in this area and the number of learners currently undertaking study in the subject at level 8 at NCI. The programme director outlined the level of benchmarking and evaluation of other programmes in the area. This section of the programme submission would benefit from greater articulation of the breadth of review. The panel was satisfied that employment and further education opportunities for graduates had been addressed in the submission.

1.2.8 Programme viability The Panel was satisfied that the programme is consistent with the provider’s mission and strategy. The panel heard the projections for student intake over a 5 year period. The panel were also presented with the information on the potential market for the programmes. The panel was satisfied that in its rationale for developing the programme that the College had addressed a real need.

1.2.9 Learner Protection NCI’s policy is that once a programme has commenced, the programme will be completed for all learners enrolled. NCI is currently in the process of arranging learner protection to allow learners to transfer to other institutions for all of its programmes as it was not previously subject to this legislation as a not-for-profit organisation. Until these arrangements are in place, NCI will place sufficient funds in a blocked account to allow refund of all fees as required as required by the Qualifications & Quality Assurance Act 2012, 65(4b), for each cohort of this programme submitted for validation until the cohort has completed its study programme and assessment or until such time as an agreement is made with two alternative providers to allow learners to transfer to a similar programme. 1.2.10 Assessment of learners

The panel is satisfied that the learners will be appropriately assessed and has requested that sample programme and module assessment strategy be

7

completed for all modules presented in the documentation as this was inconsistent. .

1.2.11 Quality Assurance Arrangements The panel is satisfied that NCI has appropriate quality assurance arrangements in place and that no new quality assurance arrangements are required for these programmes.

1.2.12 Ethics It is expected that providers will have procedures in place to ensure that any teaching and learning or research activity at any level shall be conducted in a manner that is morally and professionally ethical. The panel was satisfied that this requirement has been met in respect of the proposed programmes.

1.2.13 Programme Titles and Award Titles The Panel heard the rationale for the programme titles and accepted the programme titles as proposed and is satisfied that the programme title and award as proposed is appropriate and is consistent with QQI policy for the naming of awards. . 1.3 PART III: RECOMMENDATION/COMMENT FOR THE ATTENTION OF THE ACADEMIC COMMITTEE The panel of experts recommend the validation of the following programmes: NFQ Level Level 9 Programme Title

Certificate in Business Data Analytics

ECTS 30 ECTS Award Type Minor Award Subject to:

Council’s general conditions of approval

8

1.3.1 Conditions C1. An explicit statement is provided which guarantees that staff

with an appropriate background in statistics, and/or data science, data mining or machine learning will be appointed to the programme

C2. The access criteria for the programme is made more explicit

with regard to standard and non standard applicants

C3. Appropriate academic mentorship is put in place to advise learners intending on taking up the Programming in Data Analytics module

C4. Corrections are made to inaccuracies in the programme

schedules and documentation which were highlighted during the course of the meeting

C5. Elements of law (data protection, software licensing) are included in the programme

C6. Module descriptors should be completed with respect to

assessment links to learning outcomes and sample assessment.

1.3.2 Recommendations R1. Consolidate the use of technical platforms and ensure that

there is no overlap in this approach across modules R2. The structure of the proposed bootcamp for non standard

learners should be articulated in the documentation and the introduction of a landscape module which introduces the area should be considered.

R3. Ethics of big data should be embedded in the programme. R4. Statistics for Data Analytics

It is recommended that the learning outcomes for this module should refer to the application of techniques. The programme team may wish to consider reducing volume of subject matter covered in the course and adding an more appropriate introductory statistical text to the required reading.

R5. Data Warehousing & Business Intelligence It is recommended that both texts by Inmon & Kimball should be on the required reading. The programme team may wish to consider a reweighting of the continuous assessment to 60% due to the amount of time spent on ETL. In memory databases could be included here.

9

R6. Data Visualisation

The panel recommends the inclusion of this module and accessibility and design for visual impairment.

R7. Analytical CRM

The panel recommends that an explicit section on web analytics should be included.

R8. Data Storage & Management The panel recommends that graph databases should be included.

10

Appendix 1

Staff Met Dr Phillip Matthews, President Prof. Jimmy Hill, Vice President Academic Affairs & Research Dr Pramod Pathak, Dean School of Computing Mr. John McGarrigle, Registrar Dr. Jer Hayes, Dr. Eugene O’Loughlin Mr. Michael Bradford, Programme Director Mr. Fabian Armendariz Mr. Michael Bradford, Dr. Paul Hayes Dr. Philip McGovern Dr. Keith Maycock Dr. Anu Sahni Dr. Horacio Gonzalez Velez, Head Cloud Competency Centre Mr. Keith Brittle Dr. Alina-Madalina Popescu, Mr. Vikas Sahni Dr. Cristina Muntean Mr Paul Stynes, Vice Dean, School of Computing

11

2 Declaration of Completeness of Response by Panel Chair

3 Programme Team Response Validation of Certificate in Business Data Analytics Programme

Team Response The programme team for the proposed Certificate in Business Data Analytics would like to express their appreciation of the Expert’s Panel deliberations and feedback. The proposed Certificate in Business Data Analytics programme has undergone a set of considered amendments based on the panel’s feedback and the conditions and recommendations relating to the proposed programme. Responses to each of the conditions and recommendations made by the panel follow: 3.1 Conditions C1: An explicit statement is provided which guarantees that staff with an appropriate background in statistics and/or data science, data mining or machine learning will be appointed to the programme Response: Section 8.2 (Profile of Teaching Staff) contains an explicit statement indicating the NCI commits to appointing appropriate staff so that teaching will be conducted by subject matter experts for all modules in the programme. ‘The College commits, on a on-going basis, to ensure that teaching staff appointed to teach modules on the Certificate in Business Data Analytics programme have the required level of expertise and qualifications in the field of Data Analytics.

12

Additionally, the College will ensure that all appropriate staff appointments are made, when necessary, prior to each iteration of the programme schedule commencing in order to guarantee that teaching will be conducted by subject matter experts for all modules in the programme’ The recruitment advertisements for current positions available can be viewed at http://www.ncirl.ie/About/JobsinNCI/CurrentVacancies.aspx C2: The access criteria for the programme is made more explicit with regard to standard and non standard applicants Response: Sections 6.4.1 (Access Criteria) and 6.4.2 (Access Processes Including Recognition of Prior Learning (RPL)) have been revised to make more explicit the access criteria for standard and non-standard applicants. ‘Applicants are normally required to hold a minimum of a level 8 honours qualification (2.2 or higher) or equivalent on the National Qualifications Framework in a cognate or non-cognate discipline. The admission of persons not holding a cognate degree is very much the exception.

All applications will be assessed for entry based on their commitment towards the academic requirements for the programme. This may be assessed using a written application from the learner and/or by interview.

Given the target technical market for graduates of this programme, candidates will be required to demonstrate technical or mathematical problem solving skills as part of previous programme learning. Graduate from disciplines which do not have technical or mathematical problem solving skills embedded in their programme will need to be able to demonstrate technical or mathematical problem solving skills in addition to their level 8 programme qualifications (Certifications, Additional Qualifications, Certified Experience and Assessment Tests).

Standard applicants for the programme are those holders of technical, numerate degrees. Such applicants are likely to gain a higher ranking in any order of merit for admission to the programme. Typically, these would be applicants who have gained a minimum of a Level 8 qualification in a numerate discipline. Typical disciplines which would fall into this category would be

Computing

Informatics

Engineering

Mathematics

Physics

Chemistry

13

Additionally, applications will be considered for those who have gained a minimum of a Level 8 qualification in a programme with a significant IT component and/or significant numerate element. Typical programmes which would fall into this category could include

Management Information Systems

Accounting

Economics

Marketing

Management

Sociology

Biology

Such programmes in this second category may vary greatly in mathematical and information technology content. Consideration of these applications would be assessed by detailed examination of the subject content, assessments and syllabi of the programmes from which applicants have been awarded their primary degrees. In addition, candidate learners with qualifications in this second category of disciplines will be assessed via an interview to ensure that (in coming from a less technical or nontechnical background) they can successfully participate on the programme.

Applicants who do not fall into either the first or second categories above may also be admitted based on Recognition of Prior Learning (RPL). RPL will be assessed on the basis of a minimum of 3 years work experience and by portfolio or other appropriate method to establish their eligibility as measured against level 8 award standards (see section 6.4.2).’

C3: Appropriate academic mentorship is put in place to advise learners intending on taking up the Programming in Data Analytics module Response: Section 6.5.3 (Academic Mentorship) has been added to the document indicating that those learners wishing to elect the Programming for Data Analytics module will be advised as to their suitability to enrol in the module class. Faculty will ensure that this module is only available to those who have the required technical capabilities to participate successfully. Additionally, Section 6.5.2 (Bootcamps) has been added to the document. This section explicitly documents that bootcamp sessions in Programming and Statistics will be available for all learners prior to the commencement of Semester 1 of the programme.

14

‘Bootcamps As detailed in section 6.4.1, the programme team recognises that entrants to be programme may come from a variety of different disciplines. As some of the modules on the programme require a grounding in core technical and analytical skills, a series of bootcamps will be provided to develop/refresh these core skills. These bootcamps will take place prior to the commencement of first semester modules and will be block delivered over the course of 1 week. All enrolled students are eligible to attend these bootcamps. Bootcamps will cover the following topics:

• Programming

• Statistics

The Programming bootcamp will be of especial interest to those who have not previously gained much experience in computer programming. Additionally, for those entrants who may have previously gained experience or studied Programming and Statistics in the past, these bootcamps will offer a means to refresh prior knowledge.

Academic Mentorship In order to choose The Programming for Data Analytics module as an elective, learners should have a pre-existing level of programming skills or exhibit strong capabilities in terms of their abilities to write computer programs. The programme team will mentor and provide learners with advice as to their suitability to undertake the Programming for Data Analytics module. This module will only be available to those who have a sufficiently technical background through their previous qualifications or who exhibit the necessary technical skills during the course of the first semester.’

C4: Corrections are made to inaccuracies in the documentation and programme schedule which were highlighted during the meeting Response: The programme schedule has been reviewed and corrected where necessary. Additionally, the programme schedule has been cross-referenced with module descriptors. The documentation has been proofed to ensure that all references are to this programme only. C5: Elements of law (data protection, software licensing) are included in the programme Response: The ICT Strategy and eBusiness Implementation module (7.3.3) has been modified to incorporate the pertinent elements of Law, specifically Law – Legal issues in Data Analytics

• Evaluate Compliance (e.g. Sarbanes-Oxley, Basel lll) requirements • Demonstrate a comprehensive knowledge of Contracts and Contract Law • Sources of Privacy Law

15

• Statutory protection • Data Protection Principles • Supervision, rights, remedies, offences. • Trans-border data flows

C6: Module descriptors should be completed with respect to assessment links to learning outcomes and sample assessment Response: Module descriptors have been reviewed with assessments linked to learning outcomes. Additional sample assessments have been provided for modules for which none had previously been presented. 3.2 Recommendations R1: Consolidate the use of technical platforms and ensure that there is no overlap in this approach across modules Response: This had always been the intention of the programme team. The consolidation of technical platforms etc. is now specifically documented in Section 6.5.1. ‘To the greatest extent possible, module owners will attempt to consolidate the technological platforms and programming languages etc. that are used across modules so that learners can reinforce skills rather than be presented with with a diverse set of environments’ R2: The structure of the proposed bootcamp for non-standard learners should be articulated in the documentation and the introduction of a landscape module which introduces the area should be considered. Response: Section 6.5.2 (Bootcamps) has been added to the document. This section explicitly documents that bootcamp sessions in Programming and Statistics will be available for all learners prior to the commencement of Semester 1 of the programme. R3: Ethics of big data should be embedded in the programme Response: The Data Warehousing & Business Intelligence module now contains the topic Ethics of Big Data. (7.2.3) It was decided to place this topic in the Data Warehousing and Business Intelligence module as the module is mandatory for all learners. ‘Big Data Context

• Data warehouse and business intelligence within an ecosystem of data analytics technologies

• Handling unstructured and semi-structured data

• Integration of data warehousing with data analysis solutions via Map-Reduce processing

• Ethics of Big Data’

R4: Statistics for Data Analytics - It is recommended that the learning outcomes for this module should refer to the application of techniques. The programme team may wish to consider reducing volume of subject matter covered in the course and adding an more appropriate introductory statistical text to the require reading Response: The learning outcomes for the Statistics for Data Analytics (7.1) module have made it more explicit that application of tools and techniques are an integral part of the module. The module’s aims and objectives also more explicitly reflect this point. In terms of reducing the volume of

16

subject matter, the programme team have removed Multiple Regression from the module descriptor. An introductory text has also been added to the Required Reading list. R5: Data Warehousing - It is recommended that both texts by Inmon & Kimball should be on the required reading. The programme team may wish to consider a reweighting of the continuous assessment to 60% due to the amount of time spent on ETL. Response: Both Kimball and Inmon texts are now listed as required reading. The programme team agrees with the panel’s recommendation with respect to the assessment weightings and has made changes accordingly. (7.2.7) Allocation of marks Continuous Assessment

60%

Exam 40% Total 100% R6: Data Visualization - The panel recommends the inclusion of this module and accessibility and design for visual impairment. Response: The programme team has decided to maintain 30 ECTS for the Certificate award and have not accepted this recommendation at this time. It will be held under review after the 1st iteration of the programme. R7: Analytical CRM - The panel recommends that an explicit section on web analytics should be included Response: The Analytical CRM module has been modified to include a section on Web Analytics (7.6.3). ‘Web Analytics

Using off-site and on-site web analytics Web Usage Mining Query Log Mining Click Analytics’

R8: Data Storage & Management: The panel recommends that graph and in-memory databases should be included Response: The Data Storage & Management module (7.5.3) now contains coverage of the topics of graph databases and in-memory databases.

‘Investigate and analyse graph databases (e.g., Neo4j) and in-memory databases (e.g., SAP HANA) concepts and technologies’

17

4 Programme Schedule: Certificate in Business Analytics (30 ECTS) Name of Provider National College of Ireland Programme Title (i.e. named award) Certificate in Business Data Analytics Award Title (HETAC named award) Certificate in Business Data Analytics Stage Exit Award Title Modes of Delivery (FT/PT/ACCS/BLENDED/OC etc)

Full Time, Part Time, Blended, ACCS

Award Class Award NQF Level

Award EQF Level

Stage Stage NQF Level

Stage EQF Level

Stage Credit (ECTS)

Date Effective ISCED Subject Code

Minor 9 7 AWARD 9 7 30 September 2014 Re f

Module Title S e m

Module ECTS Credit Number

Total Student Effort Allocation of Marks Status

(M/E) NQF Level

Total Hours

Contact Hours

Independent Learning

CA %

Project % Practical %

Final %

Total %

Statistics for Data Analytics 1 M 9 10 250 52 198 50 50 100 Data Warehousing and Business

Intelligence 1 M 9 5 125 39 86 60 40 100

Strategic ICT & eBusiness Implementation

1 M 9 5 135 30 105 100 100

Programming for Data Analytics 1 E 9 10 250 52 198 20 60 20 100 Analytical CRM 1 E 9 10 250 52 198 40 60 100 Data Storage and Management 1 E 9 10 250 52 198 50 50 100 Special Regulations: Availability of elective modules will be conditional on i) timetable arrangements, and ii) that the number of students electing to take the module exceeds a given minimum number. In semester 1, learners must complete an elective module (or elective modules) that account for 10 ECTS credits per semester.

17

5 NCI Memo on Context of Validation

National College of Ireland Validation of :

Certificate in Business Data Analytics, 30 ECTS, Minor Award in association with

MSc in Data Analytics

These programmes were evaluated using QQI’s policy for devolution of validation subprocesses. The programmes were submitted to QQI in March 2014 . A desk review had not been received from QQI, however permission to proceed with the panel was given. Membership of the Expert Panel was agreed with QQI’s Programme Accreditation Unit. No member of the panel has indicated a conflict of interest and has signed a declaration to that effect. The panel considered the programmes’ submission documents, self-evaluation report. The final report was agreed by the panel. The programme team’s response to the Expert Panel’s agreed report has been considered by the panel and the Chair has indicated in the final report that the panel is happy that the conditions of the report have been fulfilled and that the programmes should be recommended to QQI for approval.

___________________________ John McGarrigle Registrar & Company Secretary

9th May 2014

18

19