royal irish academy of music - 2003 supplement to audit

8/14/2019 Royal Irish Academy of Music - 2003 Supplement to Audit

1/5

Royal Irish Academy of Music - 2003Supplement to Audit Report

Control of IT ProjectBackgroundIn early 1999 the Royal Irish Academy of Music set out to replace its studentadministration system. It was envisaged that the new system would eventually manageall aspects of student administration including registration, fee payment, examination andresults processing. It would also deal with the allocation of teachers, classes and awards.In the case of local centre examinations it would also deal with the allocation andscheduling of examination resources (examiners and venues). The system was intendedto be integrated with thefinancial management system.The main reason for installing a new system was that the Academy had reason to believeits existing system was not Year 2000 compliant. It was envisaged that part of the newsystem would be installed prior to the end of 1999 and the remainder in 2000. On 27January 1999 a total budget of 190,460 for the project to be spent over two years wasapproved by the Management and Finance Committee.The project was intended to comprise two phases. Phase I was to involve the analysisand design of the new system and design of the conversion of existing data. Phase II wasto involve the development of the student assessment system based on the designprocured during Phase I.Project ImplementationOn 31 March 1999 the Management and Finance Committee, following a tendercompetition, approved the award of a contract in respect of Phase I of the project at afixed price of 60,226 (Contract 1).On the same date the Committee approved the placing of a separate contract with thesame contractor for the development, testing and deployment of a student assessmentmodule (Contract 2). This contract was on a time and materials basis. The contractorestimated the VAT inclusive cost of this contract at 30,727 - 38,409 as a guide forplanning purposes only. The contractor promised to commit to a fixed price for thedevelopment and implementation of this module once requirements were finalised inMay.In May 1999 the project manager agreed with the contractor that the Academy'simmediate need was for a system to generate invoices to the majority of students in Julyof that year rather than anything to do with student assessments as originally proposed inContract 2. Accordingly the Academy agreed that the contractor should proceed todevelop an invoice production module in substitution for the student assessment module.On 26 May 1999 the Management and Finance Committee noted that a supplementary


2/5

contract for an estimated 21,356 in respect of an invoice production m odule (Contract 3)had been placed on a time and materials basis with the contractor. The estimate in thiscontract was for the extra costs of developing the invoice production module over andabove the estimate in Contract 2. Thus the Academy expected at that point that it wouldacquire the invoice production module for something in the region of 52,083 - 59,765.The Committee accepted assurancesfrom he project manager that the overall budget of190,460 over two years would not be exceeded. No tenders were sought from othersuppliers in respect of either Contract 2 or Contract 3.Contracts 2 and 3 involved work which was originally envisaged as part of Phase II of theproject. One consequence of the award of these contracts w as that development of partsof the new system now proceeded in parallel with the analysis and design of the newsystem. This meant that the contractor was unwilling to commit to afixed price contractbecause the amount of work needed on foot of these contracts was difficult to estimate atthe outset.Phase I of the project was duly completed in July 1999, at the agreed fixed price of60,226, with the delivery by the contractor of the analysis and design of the new systemand the design of the data conversion. The invoice production m odule was also deliveredat a cost of 118,455 or about double the cost estimated in May. The project manager'sassessment of the invoice production module at the time was that, as delivered, it did notmeet the Academ y's needs. In particular, based on tests run by the Academy, the projectmanager concluded that the module was slow, cumbersome and awkward in operationand that operators experienced difficulty in making even the smallest amendments tostudent records on the system.On 3 A ugust 1999 tenders for the remainder of Phase II were invited from fivecompanies. Two tenders were received - one for 224,298 and the other, from thecontractor who had completed Phase I, for 282,833. In evaluating the tenders theproject manager concluded that the Academy's experience to-date with the existingcontractor was no t satisfactory in that the invoice production m odule was both overbudget and had many shortcomings. The evaluation also noted that the contractor wasnot willing to give afixed price for external interfaces and post-implementation support.On 6 October 1999 the Board of Governors agreed to expend an additional 311,086 oncomputerisation. On 19 October 1999, the Academy signed a contract with the originalcontractor for 273,476 (Contract 4).In due course the contractor delivered the new student administration system and on 26April 2000 the project manager formally accepted the system on the A cadem y's behalf.At that time the total cost to the Academy of all the projects was 452,157.System R eviewNotwithstanding its acceptance by the Academy, the system proved problematicfrom heAcadem y's point of view. Accordingly, it commissioned a high level review of the


3/5

system which reported in January 2003. It found significant problems with the systemincluding The need for manual processes to patch up the system shortcomings Updating of certain data tables was resulting in progressive corruption ofhistorical data The nature of the user interface which was judged to be flawed and difficult touse.

The review concluded that the administration of the Academy was being adverselyaffected. It suggested it would not be cost effective to fix the system and thatreplacement was the better option.Replacement SystemAs a result of a detailed review by the Management and Finance Com mittee, the Board ofGovernors approved going to tender for a replacement database and approved a budget of50,000 for thefirst phase of the project. This database comprised, initially, a studentadministration system and a local centre examination system with core functionality. Thesystems are designed to have a capacity for further development to include integrationwith the Academ y's accounts and payroll systems. Tenders were invited and followingan evaluation process an agreement was signed with the selected tenderer on 20 June2003. The expected cost was 50,000 including VAT. In January 2004 the local centreexamination system became operational and the student administration system becameoperational in March 2004. Certain datafrom he old system was transferred to the newsystem but it was felt thatfinancial data on the old system was unreliable and likely tocause problems and such data was not transferred. Despite this precaution there were,initially, some problems with 'dirty data' from he old system.Further expenditure on a Phase II involving the local centre examination system wasexpected to be completed by the end of 2004 at a final cost of 15,217. Documentssetting out extensive requirements for a Phase III have been prepared. Phase III has yetto go to tender but it is estimated that costs will exceed 150,000.Audit Concerns

The project may not have been properly managed by the Academy and, inparticular, proper analysis and design of the proposed system had not beencompleted prior to installing the invoice production module Costs may not have been properly controlled The contract was placed with the same contractor for Phase II notwithstanding theproblems in relation to Phase I that had emerged at that stage The Academy accepted and paid for the system notwithstanding the fact that theAcademy appeared to have been aware of shortcomings in the system borne outby the subsequent high level review.


4/5

Director's ResponseThe Director pointed out that the original decision by the Academy to install a newsystem was taken at a time when compliance with Year 2000 was a major and urgentissue for many organisations. In 1999, the Academy, after a number of meetings with thespecial interest group set up by the Department of Education and Science, decided thatthe existing system needed to be replaced urgently and this had a bearing on the natureand type of contract entered into and, ultimately, the costs associated with the contracts.It would be unfair to assess the Academy 's actions without taking these environmentalfactors into consideration.The Academy decided that, in developing the new system, it would be important to fasttrack the implementation of the student assessment module for 1999/2000. Invoicesproduced by the new system were required by a deadline of 9 July 1999. If the old ITsystem was to cope with this requirement, some additional programming work would berequired. The decision taken by the Academy was to fast track the relevant m odule of thenew system rather than incur costs in upgrading the old system.The Director stated that the Academy - through its Management and Finance Committee,a sub-committee of the Board, and the Executive - attempted to monitor the continuingcost of the projects. The cost of Contract 3 which was on a time and materials basis,escalated in part due to the fact that changes were made to the contract specification tobring itfrom he full student assessment module to the invoice production module.In November 1999, the Academy took the decision to move the budgetary responsibilityfor the projectfrom he project manager to the Registrar in respect of Phase II to furtherstrengthen the controls.The Director informed me that the decision to accept the contractor's proposal for PhaseII was based on the contractor's familiarity with the Academy 's operations and existingsystems and that the increased costs incurred in Phase I were due primarily to the natureof the work being more extensive than originally planned. The Director expressed theview that the Academy incurred additional costs in 1999 in order to ensure that ITsystems w ould be Year 2000 compliant.The Director stated the Academy accepted and paid for the system, having been givenassurancesfrom he contractor that any problems which manifested themselves would beremedied. Whereas the system was not ideal, it was utilised to underpin the Academy'soperational and management systems for four years and is currently being used as anarchive to retrieve non-financial historic data, such as student records, exam results,alumni/other queries etc.The Director indicated that - given the somewhat unique and complex needs of theAcademy and the contextual circumstances at the time - he does not accept that theproject was not properly managed; rather, he would assert that operational adjustmentswere made, in the light of problems emerging. These appropriate and necessary


5/5

adjustments were made right through the process (through 2003 and 2004), whereoperational or management imperatives warranted them. There was some slippage incost control, due to the contractor's 'time and materials' perceived requirement of thecontract. The Management and Finance Committee, once aware of the cost overrun, kepta monthly scrutiny of progress; and, latterly, when the business case for replacing thecontractor was made, it instituted appropriate action.

John PurcellComptroller and Auditor General**> May2005