report final draft 1.3.1
TRANSCRIPT
-
8/13/2019 Report Final Draft 1.3.1
1/52
Page 1of 52
ASSESSMENT BRIEF - COVER SHEET
Course: BSc Computing Year: 1 CSY1021 - ComputingMathematics
TCA No: 2 Title: Practical Applications of Statistical Techniques
Date due out: Date due in:
21/04/2013
Extension date: Extension agreed by:
Group Name:
Group: 2 (Team Caffeine)
Group Member Names Student ID's
Dennis Wilson (Group leader) 11424939
Jonathan Nicholls 10321515
Josh Phipps 12408771
Jonathan Raine 12406672
Chris Adshead 12417844
Adam Lang
Chris Abiodun 12408576
Rhys Hall 12412314
Tutor: Mark Johnson
ASSESSMENT FEEDBACK:RATING SCALE Excellent Good Satisfactory Needs some
more work
Needs much
more work
Formulated Aims and
Objectives (10%)
Problem Domain Elicitation
Activities and Findings
(10%)
Evaluation Questionnaire -
Quality of Design and
Deployment Mechanism
Utilised (15%)
Collection and
Representation of Study
Data (15%)
Statistical Analysis of
Results
(15%)
Study Conclusion and
Recommendations
(10%)
Client Presentation (15%)
Report Quality
Presentation, format and
use of English (10%)
Specific aspects of your assignment that the
marker likes:
Specific aspects of your assignment that need more
work:
Tutors Signature: Date: Grade:
-
8/13/2019 Report Final Draft 1.3.1
2/52
Page 2of 52
ContentsAbstract ................................................................................................................................................... 3
Introduction ............................................................................................................................................ 3
Background and rationale ................................................................................................................... 3
Aims and Objectives ............................................................................................................................ 3
Research Methodology ........................................................................................................................... 4
Problem Domain Review ......................................................................................................................... 5
Identification of the Problem Domain ................................................................................................ 5
Current System Review ....................................................................................................................... 8
Comparable Virtual Learning Environments ..................................................................................... 10
Key Stakeholder Interview Summary Findings ................................................................................. 12
Questionnaire Development ................................................................................................................. 13
Data Collection & Statistical Analysis of Results ................................................................................... 14
Conclusion and Report Recommendations .......................................................................................... 25
Report recommendations ................................................................................................................. 26
Future work ....................................................................................................................................... 29
Critical evaluation ............................................................................................................................. 29
References ............................................................................................................................................ 30Appendix ............................................................................................................................................... 31
-
8/13/2019 Report Final Draft 1.3.1
3/52
Page 3of 52
AbstractAcademic Virtual Learning Environments (VLEs) have become an essential part of the modern
university, allowing students to access learning material to enhance their learning experience and is
now not just essential for disabled students, but essential for lecturers and students alike.
But because of the importance of VLEs in Universities many different systems have been developed
without any leading system within the field.This has created confusion with many universities whoare unsure with which system to choose, as an incorrect decision could hinder student learning and
henceforth, the reputability of the institution.
This report investigates if the Northampton Integrated Learning Environment (NILE) is suitable for
the University of Northampton and most importantly is effective at supporting the Universities
learning environment. The report shows the problem domains identification and review, which
includes an evaluation of NILE and comparable systems and details the development and results of a
student questionnaire.
It then shows the analysis of data collected, identifying that most problems are caused by a lack of
training rather than flaws with the system itself and listing appropriate solutions to this.
Introduction
Background and rationale
The University of Northampton received a number of formal complaints about their Virtual Learning
Environment called the Northampton Integrated Learning Environment (NILE). As ensuring student
satisfaction with NILE is essential, an investigation into these complaints was started to assess if the
NILE meets a minimum level of satisfaction with all stakeholders and is suitable for purpose. And if
not, to find reasonable solutions to any problems found.
The potential benefits of this research include:
Increased user satisfaction. Increased University reputation. Increased user productivity through higher efficiency. Higher retention of students and increased attraction to future potential students. Reduction of costs.
Aims and Objectives
The central aim for this project is to assess NILE to ensure it is fit for purpose and meets a minimum
level of satisfaction for all stakeholders, and to give a recommendation on possible solutions to any
issues identified. To achieve this several objectives have been developed, these are:
Identification of essential requirements for a Virtual Learning Environment in a University. Review of current NILE functionality. Exploration and evaluation of alternative systems. Investigation into current training and support users receive. Investigation of current user satisfaction with NILE.
-
8/13/2019 Report Final Draft 1.3.1
4/52
Page 4of 52
Analysis of feedback to identify problematic areas. Analysis of problems to identify solutions. Show conclusions in both a report and presentation to the senior management team.
Research MethodologyTo ensure NILE is suitable for purpose, the following research methodology has been formulated:
1. Formulation of the aims and objectives.2. Investigation into the problem domain.
i. Analysis of the current software.ii. Research and analysis of alternative systems.
iii. Development of a questionnaire aimed at students to gain primary data of studentviews.
iv. Distribution of questionnaire to a sample group to test suitability and effectivenessof the questionnaire.
v. Analysis of sample survey feedback and implementation of improvements from theanalysis.
vi. Distribution of improved survey to the student user group.vii. Interview to gain primary data of IT managersview.
3. Analysis of investigation.4. Conclusion and report recommendations.
Formulation of the aims and objectives is essential to ensure the investigation remainsclear and
focused. Analysis of the current system is needed to be able to gain an understanding of NILE to
guide the investigation forward. Research into alternative systems will provide a comparison to NILE
to determine if NILE is the most suitable VLE for the University of Northampton. Using the data
gathered from the current system and alternative systems review, a questionnaire will be created to
access student satisfaction get primary data from the student stakeholder group. A sample survey
will be sent to a small group of the student stakeholders to increase effectiveness of the survey. The
interview with the IT manager will be used to gain more primary data from the stakeholders. Using
all data gained from the stakeholders, the problematic areas will be identified and solutions to these
problems will be created.
A student survey is used to gain a broad range of views, interviews could be held with the creators of
the formal complaints but this would skew the view heavily in their favour and this data would not
necessarily represent the average stakeholders view.
-
8/13/2019 Report Final Draft 1.3.1
5/52
-
8/13/2019 Report Final Draft 1.3.1
6/52
Page 6of 52
new/old users?
Does the system need to have consistent pagesor can each page be different?
If not, how different can each page be? When are prime times for user login? Do extra resources need to be spent during
these times?
Where can users access NILE? From campusonly, home?
Do smartphones need to be compatible withNILE?
What file types need to be uploaded? Will there be an upload speed cap? What is the capacity for files?
This information could be obtained by: A review of all web-board support
manuals/paper-based documentation/online
documentation.
Interviews with product suppliers. Interviews with relevant technical experts (e.g.
the IT support staff that set-up and configured
the system for use.)
Current Staff [Leaders of modules]
What experiences do the current staff havewhen using NILE?
What features of NILE do the current staff like /use?
What features of NILE do the current staffdislike / dont use?
What features would the current staff like tosee added to NILE?
What training do the current staff receive? Do the current staff use NILE to its full
potential?
Do the current staff actively use and promotethe tool to their students?
Have the current staff used another VLE before,and if so what is their opinion of it compared
with NILE?
Would the current staff accept a new system?This information could be obtained by:
Interview carried out with a broad selection ofstaff including other organisations such as CfAP,
through interviews
-
8/13/2019 Report Final Draft 1.3.1
7/52
Page 7of 52
Once a provisional understanding of the user-groups attitudes towards the web-board had
been established, a more extensive review of
staff attitudes could be achieved via a large
scale distribution of a well-crafted
questionnaire
Current students
What experiences do the current student usergroup have?
What features of the current system do theylike or use?
What features would they like to see added? What features do they dislike or dont use? Have they had difficulties using or
understanding NILE?
Does the user group use NILE to its fullpotential?
What training do the current students have? Is this training good enough for the current
students to understand and use NILE to its full
potential?
This information could be obtained by:
Interviews with the users who have expresseddissatisfaction with NILE
Once a provisional understanding of the user-groups attitudes towards the web-board had
been established, a more extensive review of
student attitudes could be achieved via a large
scale distribution of a well-crafted
questionnaire.
The key system stakeholder
A review of the underlying aims and objectivesof the university management team (specifically
towards the current web-board) and their
rationale for having a web-board system within
the university.
The key stakeholders expectation for any web-board system employed within the university.
This information could be obtained by:
Interviews with key project stakeholders.
Comparable systems
A review of other comparable software systemson the market leading to a comprehensive
understanding of the features and functionality
common to virtual integrated environment
To establish if other systems provide better orimproved functions than NILE, including but not
-
8/13/2019 Report Final Draft 1.3.1
8/52
Page 8of 52
limited to:
Do alternative products have features NILEdoes not have?
If so, are these features important / needed? Does our product have features they do not? If so, are these features redundant? Are these alternative products easier for the
user, such as simpler to understand and use?
Are these products better technically such asfaster downloads?
Do these products have better reviews?This information could be obtained by:
Background reading (developerliterature/company web-sites) outlining
product specifications. Possible interviews with alternative product
technical and marketing teams.
Relevant legislation
A review of any relevant legislation relating tothe use of the software within the university
(Disability Discrimination Act, Data Protection
Act, etc.)
This information could be obtained by:
Possible interviews with relevant subject andlegal experts either from the University of
Northampton or hired independently.
Current System Review
NILE is provided for by Blackboard and has been built for a wide range of Universities, it includes
several embedded tools which create modules and pages for the modules and a collection of
features for academic institutes. Blackboard provides an extension program which allow certain
clients to create and customise NILE with add-ons, these are shared so can be used by other
intuitions. The catalogue of add-ons is large and range from support and helpdesk software to voice
authenticators although some of these add-ons require payments.
-
8/13/2019 Report Final Draft 1.3.1
9/52
Page 9of 52
The current system features (Blackboard, [n.d]) include:
A Calendaro This function allows the user to track important dates within the university semester or
add their personal appointments or obligations.
Online grading and assessmentso This feature allows the user to receive feedback on modules via NILE providing fast
response times for grades given to them by module tutors.
Student activity reportso This allows staff to monitor student activity by being able to see what areas of NILE
certain students are accessing ensuring they are keeping up to date with the work.
Social networko Users are able to create a profile in order to interact and stay in contact with their peers.
Survey toolso This lets university staff or students to create one or more surveys that can be
distributed online to other university staff or students.
Institutional hierarchyo This allows the University to separate NILE into different subsections which allows staff
members to have different levels of administrative control as well as the ability to
customise each subsection to its specific need.
Blogso Students or staff may create a blog related to their course, assignments or university life.
Doing this allows other students to keep informed about several matters.
Announcements / Notificationso Module leaders are able to post notifications on NILE. When their students access the
related subject they are initially greeted with the notifications section in order to see
any important changes or updates.
Individual customisations (for both student and course)o Users are able to personalize their own NILE interface. This includes being able to
prioritise different sections by placing them in different areas on the screen. Topics that
the user deem more important can be placed at the top of their page.
Mobile and Smart Device applicationso There is a NILE application available for iOS and Android, this has a variety of features
including a satellite navigation system for the university, access to grades, access to
timetable of students, a news feature, limited library features and listing for events in
the near future.
-
8/13/2019 Report Final Draft 1.3.1
10/52
Page 10of 52
Email updateso Automated emails sent daily to users of NILE to inform users of new announcements.
Users are able to opt out of this by unsubscribing to this feature.
In-site support featureso Allows the University to add its specific unique support features into NILE.
Wikio Module leaders are able to create a wiki related to the course. Students are able to edit
and update this, contributing to the data stored.
Search Systemo The system can be searched for relevant modules.
The University of Northampton has provided several text, PDF and video tutorials which are
available from the Help tabfor both guests and users. These tutorials include:
Logging into NILE Changing your password and personal information. Customising your NILE Homepage Downloading files from NILE Printing files from NILE Sending email through NILE Getting started with NILE Using the tools in NILE Assessment in NILE ePortfolios in NILE Other useful guides
Comparable Virtual Learning Environments
An investigation into comparable VLEs was started to potentially find a more suitable VLE than the
current system. This investigation would provide information to be used to compare and contrast
against the current system to allow a more accurate assessment of NILE.
-
8/13/2019 Report Final Draft 1.3.1
11/52
Page 11of 52
Table 2Comparable systems features list(Moodle, 2011; Sakai [n.d.]; SourceForge [n.d.])
Feature NILE
(Blackboard)
Moodle Sakai Bodingtons
Calendar Yes No No No
Online grading and
assessments
Yes Yes Yes Yes
Student activity reports Yes No Yes No
Social network Yes No Yes No
Survey tools Yes Yes Yes Yes
Institutional hierarchy Yes No No No
Blogs Yes No No No
Notifications Yes No No No
Individual customisations
(for both student and
course)
Yes Only the
course page
can be
customized
No No
Mobile and Smart Device
Applications (Apps)
Yes No No No
Email Updates and
Notifications
Yes No Yes No
Announcements/News Feed Yes Yes Yes No
In-site Support Features Yes No No No
Search system Yes Yes Yes No
Wiki Yes Yes Yes No
Group work No Yes Yes YesPeer feedback tool No No No Yes
Instant Messaging No Yes Yes No
Course file search No Yes No No
Voice tools No No No No
Course glossary No Yes No No
Journals No Yes No No
Podcasts No No Yes No
Question creations No Yes No No
Detailed user profiles No Yes No No
Research suggested that if it is viable to replace the current system, then the VLE called Moodle
would be heavily considered. This is for a number of reasons; firstly being that it is open source. This
will cut costs down significantly along with the need for additional technical support staff. Secondly,
the features in Moodle are more refined and developed because it is open source and this means
that the features offered are made specifically because there is a need for it. Referring to the table,
it is seen that Moodle has integrated features which are more refined and sporting more modern
tools such as instant messaging and discussion forums for users to exchange ideas and to discuss
topics. Thirdly, it is very commonly used amongst colleges and some universities so its functionalityis proven. Other comparable systems will also be considered, such as Sakai. This is a less commonly
-
8/13/2019 Report Final Draft 1.3.1
12/52
Page 12of 52
used system which is used by other colleges and universities. It also sports integrated features
similar to Moodle, but research suggests that it is more influenced by social networking websites
than being academic in nature. In addition, it isnt open source and it has to be funded which could
take its toll on the budget.
The last VLE to be considered was called Bodington. This features similar tools as the systems
described above such as instant messaging, user logins and forums. It is one of the first VLE to bedeveloped, but as of today it is no longer in development. Henceforth this system cannot be
seriously considered any further. No development of any system means that it will become outdated
quickly and will be exposed to security issues.
Key Stakeholder Interview Summary Findings
An interview with the IT manager was conducted to gain primary data from the IT, the following
statements are extracts from the interview transcript that were deemed the most important to the
investigation:
Q2) Have the staff received the same training for the use of NILE as the students?
The training that we give at the moment depends on the school that the members of staff are in.
We, as a university policy offered all staff the opportunity to come to some training sessions which
were quite comprehensive to show them more than just announcements which is the general
information that is available at the moment. So we set up a situation where schools could nominate
staff members to come to the training. Unfortunately, certain schools didnt take up our offer so
there is a real difference between the amount of training that certain schools did not send their
staff, and I personally think that that may reflect on the quality of service that they are giving
because they do not know about the opportunities, but yes, we did try to train them.
Q4) What is the most common problem students have?
I thought youd ask this question so I spoke to the frontline support staff and the most common
problem that students have is not knowing how to use the tool, so people come to us, and people
dont actually understand what to do, so for example, we had a student who wrote and official
complaint saying that they couldnt access their grades, when in fact that wasnt true. Their grades
were available, but they didnt know where to look in the tool to find their results. It is usually
misinformation where students complain because they dont really know how the tool works.
Q5) Are there constant updates on NILE or just at request?
Blackboard is the company that actually we pay for their service, and we get updates from them on a
sporadic basis, if they have dont some new updates or added new functionality then theyll tell us
you pay for this service, in your licence agreement, well send you the updates. We ourselves in
the IT team dont do any adjustments to their code, so we use the tool as they provide it, but yes,
there are updates and it is very sporadic, sometimes, in the peak period, had one time where there
was about 3 updates in a month, typically, there is one update every 6 months, but it does vary and
it is entirely down to them and their developers in the team, who tells us when updates are
available.
-
8/13/2019 Report Final Draft 1.3.1
13/52
Page 13of 52
Q6) What are the main objectives of NILE?
For us, NILE ticks two boxes. It complies with the legal regulations that we have including about
providing the material in an electronic format for all people including people with disabilities, so we
have a legal requirement under the equality act and the provision of information in higher education
that is part of the equality act, previously called the disability discrimination act, so it is important
that we comply with that and for me, the most important one, the second one, is providing a good
quality of service for students. Ultimately, we want to make sure that students enjoy their time here
at the University of Northampton that students can find the information they need, and its
absolutely imperative that I believe that NILE is used to its full potential in order to meet the second
criteria of student satisfaction and experience.
Q11) The University of Northampton is a business organisation. Would you consider that to
improve NILE you would have to invest money in blackboard as a business? In the market rightnow there are open source systems that can be improved, are there facilities in the university that
could do that?
Absolutely, there is already the two sums of money that I have previously mentioned which show
that we are committed. The 250,000 for support and 180,000 for the licence show that we take it
very seriously. If you prove that what we are offering at the moment isn't good enough then I would
press for at the top levels such as the vice chancellor to ask for more money, I think that it is very
important. If you can prove that NILE as it is isn't working then 100% you have my guarantee that I
would take it to the management top level requisites of this institution and tell that that because it
isn't working, we need another solution. So I am open, even though it would be very hard as wehave had a relationship with blackboard for a long time, if you could prove with your investigation
then you will have my commitment to making changes.
Questionnaire DevelopmentUsing the problem domain review, a list of questions to ask students was created and a basic paper
based survey was made. As online survey would improve efficiency by increasing distribution and
analysis, potential online survey tools were investigated.
KwikSurvey was chosen as the survey tool used as this allowed logical sequencing and unlimitedquestions. Logical sequencing allows more specific questions to be asked without asking questions
that are not applicable to the user, and unlimited questions was needed as most survey tools limited
the questions to ten when an estimated twenty questions was needed.
The survey was distributed to six students as a sample to confirm the survey effectively gathered
feedback. Five completed it with no issues and had no suggestions or feedback to give and the sixth
student gave feedback of:
Question 23 was probably the most difficult to fill out as some features would not affectme.
Didn't really notice them[the heading text] until they were pointed out.
-
8/13/2019 Report Final Draft 1.3.1
14/52
-
8/13/2019 Report Final Draft 1.3.1
15/52
Page 15of 52
11 Is there a relationship between the
number of training hours a student
receives and the percentage of
students who have not had technical
difficulties
Correlation R = 0.683088407 N
12 Is there a relationship between the
number of training hours a student
receives and percentage whomanaged to resolve the issue
Correlation R = 1 But data is too
small to draw a
definitiveconclusion
N
To find the relations, certain questions had to be converted into weighted averages as they used a
non-numerical scale. All of the weighted averages use the same; 2 is strongly agree with the
statement, 1 is agree, 0 is neutral, -1 is disagree and -2 is strongly disagree. Below are all the graphs
of the relations:
Graph 1Training hours compared with Average number of problems a year
0
0.5
1
1.5
2
2.5
3
0 1 2Averagep
roblemsayearperstudnet
Training hours
Training hours vs Average problems a
year
-
8/13/2019 Report Final Draft 1.3.1
16/52
Page 16of 52
Graph 2Training hours compared with satisfaction
Graph 3Type of training compared with satisfaction
0.75
0.8
0.85
0.9
0.95
1
1.05
0 1 2
Satisfac
tion
Training hours
Training hours vs Satisfaction
0.92
0.94
0.96
0.98
1
1.02
Talk from lecturer IT Specialist Video Web-based
Satisfaction
Type of training
Type of training vs Satisfaction
-
8/13/2019 Report Final Draft 1.3.1
17/52
Page 17of 52
Graph 4Training hours compared with easy to use
Graph 5Average number of problems a year compared with satisfaction
0
0.2
0.4
0.6
0.8
1
1.2
None 1 hours 2 hours
Easeofuse
Hours of training
Training hours vs easy to use
0
0.2
0.4
0.6
0.8
1
1.2
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Satisfa
ction
Number of problems
Average number of problems a year vs
Satisfaction
-
8/13/2019 Report Final Draft 1.3.1
18/52
Page 18of 52
Graph 6Average number of problems a year compared with if NILE helped the students learning
Graph 7Year of study compared with satisfaction
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 2 4 6 8 10 12 14
NILEhelpedmylearning
Average number of problems a year
Average number of problems a year vs Nile
helped my learning
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1 2 3 4
Satisfaction
Year of study
Years of study vs satisfication
-
8/13/2019 Report Final Draft 1.3.1
19/52
Page 19of 52
Graph 8School compared with Average number of problems a year
Graph 9School compared with NILE usage every week
0
2
4
6
8
10
12
14
School of the
Arts
Northampton
Business
School
School of
Education
School of
Health
School of
Social Science
School of
Science and
Technology
Averagenumbero
fproblmsayear
School
School vs Average number of problems a
year
0
1
2
3
4
5
6
School of the
Arts
Northampton
Business
School
School of
Education
School of
Health
School of
Social Science
School of
Science and
Technology
NIleusageeveryweek
School
School vs Nile usage every week
-
8/13/2019 Report Final Draft 1.3.1
20/52
Page 20of 52
Graph 10Type of training compared with average number of problems a year
Graph 11Nile weekly usage compared with average number of problems a year
0
2
4
6
8
10
12
14
Talk from lecturer IT Specialist Video Web-based
Typeoftraining
Type of training
Type of training vs number of problems
0
2
4
6
8
10
1 2 4 5 6 or moreAveragenumberofprblemsayear
Weekly useage
Nile weekly usage vs Averge number
of problems a year
-
8/13/2019 Report Final Draft 1.3.1
21/52
Page 21of 52
Graph 12Nile weekly usage compared with satisfaction
Graph 13Alternative VLE statements
-1.5
-1
-0.5
0
0.5
1
1.5
0 1 2 4 5 6 or moreSatisfaction
Nile weekly usage
Nile weekly usage vs satisfaction
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Nile is easier to use Nile has better
functions
Nile has helped me
more
Levelofagreementwiththestatement
Statements
Alternative VLEs statements
Moodle
Other
-
8/13/2019 Report Final Draft 1.3.1
22/52
Page 22of 52
Graph 14Year of study compared with average number of problems a year
Graph 15School compared with satisfaction
0
2
4
6
8
10
12
14
1 2 3 4Averagenumberofproblmesayear
Year of study
Year of study vs Average number of
problems a year
0
0.2
0.4
0.6
0.8
1
1.2
1.4
School of the
Arts
Northampton
Business
School of
Education
School of
Health
School of
Social Sciences
School of
Science and
Technology
Satisfcation
School
School vs Satisfaction
-
8/13/2019 Report Final Draft 1.3.1
23/52
Page 23of 52
Graph 16Type of student compared with satisfaction
Graph 17Potential features compared with agreement with statement
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
International student Non international student
Satisfcation
Type of student
International vs satisfication
0 0.5 1 1.5
A section dedicated toCourse file searchQuestion creation
Peer feedback toolCourse blogs
Course glossarySurvey tools
Instant messagingJournals
Social networkingPodcads
Detailed user profilesVoice tools
Weighted Average
Featuresw
anted
Which features would you like to see
added to Nile?
-
8/13/2019 Report Final Draft 1.3.1
24/52
Page 24of 52
Graph 18Current features compared with percentage of users who dont use the feature
Graph 19Current features compared with agreement with statement
0 20 40 60
Module annoucements
Email notifications
Personalized pages
SmartPhone app
Discussion boardsCourse blogs
Calendar
Nile search engine
Course wikis
Percentage of users who don't use the feature
Feature
Percentage of users who don't use
feature
0 0.2 0.4 0.6 0.8 1 1.2
Module annoucements
Email notifications
Personalized pages
SmartPhone appDiscussion boards
Course blogs
Calendar
Nile search engine
Course wikis
Weighted average for the feature
Feature
Which features are liked, disliked or
not used?
-
8/13/2019 Report Final Draft 1.3.1
25/52
Page 25of 52
Graph 20Training hours compared with NILE helped improve my learning
Graph 21Training hours compared with percentage of students who have not had technical
difficulties
Conclusion and Report RecommendationsThe following information was gained from the IT manager:
Certain members of staff do not accept training Students typically misunderstand the tool Blackboard is updated on average every 6 months sometimes with new features The IT team do not change the code of Blackboard
0
0.2
0.4
0.6
0.8
1
1.2
0 hours 1 hour 2 hoursNILEhelpedimrpove
mylearning
Training hours
Training hours vs NILE helped improve
my learning
0
20
40
60
80
100
120
0 hours 1 hour 2 hoursPercentwhohavenothadtechnical
difficu
lties
Training hours
Training hours vs Percentage who
have not had technical difficulties
-
8/13/2019 Report Final Draft 1.3.1
26/52
Page 26of 52
Correlations have been found between the following:
Training hours and average problems a year Training hours and satisfaction Training hours and if users found NILE easy to use How often a user has a problem with NILE and their satisfaction with NILE Average number of problems a year and if users felt NILE helped their learning Students weekly usage of NILE and their satisfaction Students year of study and their average number of problems a year Number of training hours a student receives and if they thought NILE helped improve their
learning
Report recommendations
Recommendation summary:
Increase training hours to two hours as a minimum Train two times a year for students of all years Make training compulsory for both teachers and students When there is a large update for blackboard update the lecturers with any new features of
that update
Create a video tutorial and use this for the training, as well as sending certain schools extrasupport
Initial Analysis of the results show the number of problems students experienced have had a direct
impact on both performance and enjoyment as shown by the direct correlation between averagenumbers of problems a year, compared with student satisfaction and if students found NILE helped
with their learning. This could negatively affect student performance and the reputation of the
University.
Further analysis of the results indicates that many issues students have with NILE are
misunderstandings of the product rather than flaws in the system itself. This is supported by 90.9%
of students being able to resolve their issues with NILE, the strong correlations between training
hours, compared with satisfaction and ease of use, as well as the strong negative correlation
between training hours and number of problems. Because of this and statements two and fourfrom
the IT manager suggesting this, improvements to training is a reasoned response to improve usersatisfaction with NILE.
The results show a greater increase of satisfaction and ease of use from zero training hours to one,
than one training hours to two training hours. This suggests that even a small amount of training is
required for students to gain a satisfactory understanding of NILE. Although an increase of zero
training to one hour training shows an increase in the number of problems, this is likely because
students are using more advanced features of NILE which have not been fully explained to them and
this would explain why a large reduction in the number of problems is found at two training hours.
This suggests that training should be compulsory and two hours is required as a minimum for
students to fully understand all features of NILE that are available for them to use.
-
8/13/2019 Report Final Draft 1.3.1
27/52
Page 27of 52
The relations between training hours compared with satisfaction and ease of use have a logarithmic
regression of y = 0.1267ln(x) + 0.866 and y = 0.1702ln(x) + 0.8254 respectively, this equals a
weighted average of 1.0052 and 1.01238 respectively for 3 training hours. Although this is an
increase, the benefit is quite small and will likely be affected by diminishing returns which will
decrease the benefit further. Therefore two hours training should be sufficient for students.
Graph 22 - Logarithmic regression for training hours compared with ease of use
Graph 23 - Logarithmic regression for training hours compared with satisfaction
The student survey results show that the most effective at reducing average number of problems a
year is video training and video training shares the highest satisfaction. This would suggest the mosteffective and efficient way would be to create a universal video tutorial for students by using the
0
0.2
0.4
0.6
0.8
1
1.2
None 1 hours 2 hours 3 hours
W
eightedaverage
Training hours
Prediction for Training hours vs easy to use
0
0.2
0.4
0.6
0.8
1
1.2
None 1 Hours 2 Hours 3 Hours
WeightedAverage
Training hours
Prediction for Training hours vs Satisfaction
-
8/13/2019 Report Final Draft 1.3.1
28/52
Page 28of 52
current video tutorials for students to use during training and shows all features which is currently
lacking from current student support.
The School of Health showed the highest level of average number of problems followed by the
Northampton Business School and School of Education, and so these would greatly benefit from
additional support such as an IT Specialist.
The results show a high correlation between year of study and average number of problems and no
correlation has been found between weekly usage of Nile and number of problems as well as year of
study and satisfaction. This would suggest that training is required for all levels as most problems are
not resolved easily with experience, but are not flaws with NILE because 90% of all problems were
resolved. Training twice a year and for all years will allow users to be updated for new features and
features more suited for later years.
Five features had over 35% not used, although some features are likely not used due to personal
preference, most are likely not used because staff do not promote those features therefore students
do not use them. Together with the IT managers statement twoin which he criticized certain
members of staff not accepting training, training should be made compulsory for staff members to
ensure they utilize NILE to its full potential. In addition to this, the IT managersstatement five
stated that the updates to NILE where on average every 6 months, as such to ensure staff are
knowledgeable about new features, small training sessions for staff should employed after every
large update.
As the NILE usage increases, so does satisfaction and the number of problems, this indicates that
NILE has good features which users want but these features have not been fully trained and so
problems occur. This would indicate that more features are not required but could further increase
satisfaction and performance. And as 76% of students who completed the survey agreed or strongly
agreed that NILE helped their learning NILE is mostly suitable for the University of Northampton.
Only three features are widely wanted by users; a section dedicated to group work, a question
creation and a course file search. The features are offered by Moodle but these features wouldnt
justify the cost and effort of changing to a different system.
Moodle was the most highly considered comparable VLE but the student survey results shows that
most students who have used Moodle in the past prefer NILE. And as the University of Northampton
does not employ coders as stated by the IT manager answer five, Moodles open source strength
would not be fully utilised. Because of this, Moodle is not a suitable replacement to NILE.
The Sakai and Bodingtons systems which were the other comparable VLEs investigated into, were
also found to be unsuitable for the University of Northampton.This is because both of the systems
do not offer anything unique, and as no student who answered the survey had used either system,
no appraisal could be made.
Because of this, no other VLE has been found to be more suitable for the University of Northampton
than NILE.
-
8/13/2019 Report Final Draft 1.3.1
29/52
Page 29of 52
Future work
Although no other VLE was found to be more suitable, students did slightly prefer an unknown
system, the survey tool did not allow an other text field to be used so this system was not
identified. Further investigation would be needed to identify and assess if the unknown system is
more suitable for the University of Northampton than NILE.
Although training should improve user satisfaction with NILE, it may not be the best system available
for the University of Northampton. Further investigation is needed to identify the other VLE that
students preferred over NILE, as this might be a more suitable system.
Increased survey distribution for more accurate and reliable results.
Three areas of the problem domain were not investigated into; the key system stakeholder, the
current staff and relevant legislation. To increase the recommendations accuracy an investigation
needs to be started for each of these domains. This could be achieved by modifying the student
survey and using it to gather staff feedback, a legal expert to be hired either from the University or
independently who would ensure that the system is legal in the UK and interviews with the key
system stakeholder to ensure there requirements corresponds with the recommendations solution.
International satisfaction was found to be over half the satisfaction of non-international students,
further investigation is needed to ensure NILE is also suitable for international students.
Critical evaluation
The student survey which forms the main support of the conclusion has a very low participation ratefor the size of the University. For more reliable results 10% of the student population would be
needed which is an estimated 1039 (justcourses.com, n.d) students. This has caused potentially
anomalous results such as with the best type of training where only one person replied with video
causing it to have a skewed result.
The student survey did not ask the type of problem or more information about the problems users
had with NILE. A potential error with the questioning of the survey occurred where several students
answered they had technical difficulties with NILE yet later they also stated they never have had
problems with NILE. This limited the analysis and accuracy of the recommendation. Other problems
with the survey included:
Several grammatical errors Several general errors Weak scales Several important questions are not required Didnt account for students with disabilities No questions about support documentations
The distribution method would also need improving, as method used was to get as many results as
possible but at the cost of a proportioned distribution e.g. between gender and schools. This could
be accomplished by simply targeting certain areas of the campus which are specific to the criteria.
-
8/13/2019 Report Final Draft 1.3.1
30/52
Page 30of 52
Blackboard (the creator and provider of NILE) has several different licences which have different
levels of features depending on the price. Because of the sensitive nature of NILE the licence was not
disclosed as stated by the IT manager on statement Q11. As such , the investigation was not able to
differentiate between features the current system has and features which required a higher licence.
The investigation into the comparable systems was limited to only the internet because of the short
time period of the research. As such, alternative investigation methods such as direct contact withmarketing teams and direct use of comparable systems were unable to be employed which limited
the information gained and used. And the research into bodingtons was weak because it has been
discontinued and little documentation has been created for it.
To reduce research costs, a free survey tool was used, this meant that certain features were unable
to be used in the survey such as the other text field. This caused weakness in the survey such as
when asking for the student which alternative VLE they had used.
The investigation was not allowed to contact University staff, and as the investigation didnt have
the funding to hire an independent legal expert, the current staff, key stakeholder and relevant
legislation problem domains were not fully explored which has restricted the reports
recommendation.
Background reading of relevant legislation could have been employed but this would have not given
the sufficient accuracy needed, as such a subject expert would be required.
Research into both NILE and the comparable systems was insufficient, more detail should have been
investigated and large sections should have been separated into subsections such as with the
assessments, which could include online assessments, quizzes and into features for students and
features for staff. Research into old documentation caused an error where certain features where
doubled so NILE both had and did not have the feature.
References
Just courses. [n.d.] University guide University of Northampton [online]. Sine nomine. Available
from:http://www.justcourses.com/Universities/University_of_Northampton.html[Accessed on 3rd
of April 2013]
Moodle. (October 2011) Features MoodleDoc [online]. MediaWiki. Available from
http://docs.moodle.org/20/en/Features[Accessed on the 3rd of April 2013]
Sakai. [n.d.] Sakai features[online]. Sine nomine. Available from:
https://sakai.win.tue.nl/library/content/gateway/features.html[Accessed on the 3rd of April 2013]
SourceForge. [n.d.] Bodington[online]. Sine nomine. Available from:
http://sourceforge.net/projects/bodington/[Accessed on the 3rd
of April 2013]
Blackboard. [n.d.] Blackboard Learn | Blackboard Learn Platform Feature Showcase [online].
Blackboard. Available from: https://www.blackboard.com/Platforms/Learn/Products/Blackboard-
Learn/Features.aspx[Accessed on the 7th of February 2013]
Blackboard. [April 2010] Blackboard Learn, Release 9.1 New Features [online]. Blackboard. Available
from:http://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdf[Accessed on the7th of February 2013]
http://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://www.justcourses.com/Universities/University_of_Northampton.htmlhttp://docs.moodle.org/20/en/Featureshttp://docs.moodle.org/20/en/Featureshttps://sakai.win.tue.nl/library/content/gateway/features.htmlhttps://sakai.win.tue.nl/library/content/gateway/features.htmlhttp://sourceforge.net/projects/bodington/http://sourceforge.net/projects/bodington/https://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttp://www.blackboard.com/resources/learn/Bb_Learn_91_WhatsNew.pdfhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttps://www.blackboard.com/Platforms/Learn/Products/Blackboard-Learn/Features.aspxhttp://sourceforge.net/projects/bodington/https://sakai.win.tue.nl/library/content/gateway/features.htmlhttp://docs.moodle.org/20/en/Featureshttp://www.justcourses.com/Universities/University_of_Northampton.html -
8/13/2019 Report Final Draft 1.3.1
31/52
Page 31of 52
Appendix
Item 1 - Questions for the IT Manager
How many complaints have been made in the last 6 months? What complaints have students been making? Has staffs made any complaints? If so typically what complaints have staffs been making? How often on average every week does someone contact the IT department requesting help? What are the most common problems students have? What web-board training do IT services provide for students and staff using the software? What support documentation is available for the software? Is this support documentation enough? I.e. does it actually help students solve problems? Do students/staff know about these documents? Do students/staff often use these documents? What opinions of the current system do the IT staffs have? Would the IT staff like to see a new software system utilised? Do the IT staffs have any recommendations of a new VLE to use?
Item 2 - Complete IT Manager transcript
Q1) Would there be the possibility to allow downtime during the transitions?
Any products that we have, have to be available to our students pretty much 24/7, 365. So, if there
was to be a new development that involved a transition then we would have to take steps, possibly
parallel implementation where the new system is brought in over the old one, but we really cannot
afford to drop our quality of service. If there was to be any downtime, then it would likely be, with
the students in mind, I would suggest it being on Sunday, at 3 in the morning, in the off-peak times
where we are unlikely to experience any traffic. But the long and the short of the answer is that wewould expect that the service would be available 24/7, 365.
Q2) Has the staff received the same training for the use of NILE as the students?
The training that we give at the moment depends on the school that the members of staff are in.
We, as a university policy offered all staff the opportunity to come to some training sessions which
were quite comprehensive to show them more than just announcements which is the general
information that is available at the moment. So we set up a situation where schools could nominate
staff members to come to the training. Unfortunately, certain schools didnt take up our offer so
there is a real difference between the amount of training that certain schools did not send their
staff, and I personally think that that may reflect on the quality of service that they are giving
because they do not know about the opportunities, but yes, we did try to train them.
Q3) What is your role in the management of NILE?
I am the top level manager and I have a brief from the institution that ultimately, I have to provide
quality of service to students throughout the year. I have a budget for which is currently a 250,000
which pays for the IT staff who support NILE. So the 250,000 is for servicing NILE and does not
include the cost of actually buying the product, and us purely for the IT support staff who work on
NILE. Again, I am not sure that were are really getting value for money at the moment, and I am
really interested to find out the data that youve got to say whether we should be going in a
different direction, but ultimately, the book stops here when it comes to NILE.
-
8/13/2019 Report Final Draft 1.3.1
32/52
Page 32of 52
Q4) What is the most common problem students have?
I thought youd ask this question so I spoke to the frontline support staff and the most common
problem that students have is not knowing how to use the tool, so people come to us, and people
dont actually understand what to do, so for example, we had a student who wrote and official
complaint saying that they couldnt access their grades, when in fact that wasnt true. Their grades
were available, but they didnt know where to look in the tool to find their results. It is usually
misinformation where students complain because they dont really know how the tool works.
Q5) Are there constant updates on NILE or just at request?
Blackboard is the company that actually we pay for their service, and we get updates from them on a
sporadic basis, if they have dont some new updates or added new functionality then theyll tell us
you pay for this service, in your licence agreement, well send you the updates. We ourselves in
the IT team dont do any adjustments to their code, so we use the tool as theyprovide it, but yes,
there are updates and it is very sporadic, sometimes, in the peak period, had one time where there
was about 3 updates in a month, typically, there is one update every 6 months, but it does vary and
it is entirely down to them and their developers in the team, who tells us when updates are
available.
Q6) What are the main objectives of NILE?For us, NILE ticks two boxes. It complies with the legal regulations that we have including about
providing the material in an electronic format for all people including people with disabilities, so we
have a legal requirement under the equality act and the provision of information in higher education
that is part of the equality act, previously called the disability discrimination act, so it is important
that we comply with that and for me, the most important one, the second one, is providing a good
quality of service for students. Ultimately, we want to make sure that students enjoy their time here
at the University of Northampton that students can find the information they need, and its
absolutely imperative that I believe that NILE is used to its full potential in order to meet the second
criteria of student satisfaction and experience.
Q7) If it comes back that in the research that there is an alternative system that would be bettersuited to the universitys needs, would you be open to exploring other systems? And if so, are
there any constrains on that such as budgets and features?
If you could prove to me that NILE as it is, is not fit for purpose, then I would be open to
recommendations to take the product forward to either get a new product, or improve what we
have got for more services with blackboard and extra information, then yes, I would consider
changing system. It would be difficult because obviously, we have used NILE for a long time, weve
used blackboard, but I think ultimately, if you could prove that student satisfaction was really poor,
that wasnt something to do with training or it genuinely is a poor product, then we would. In terms
of the second question, which related to budgets, we have the 250,000 which is for the support at
the moment, and we pay 180,000 every 24 months to blackboard. At the moment, we are tied in
until September this year (2013), so any funds if you suggested another approach, we obviouslywouldnt renew the contract and that 180,000 could be used for a new product.
Q8) As IT manager, when complaints are made, and improvements are suggested, though you are
a key figure within the domain, are you aware of the problems and do you suggest solutions
yourself, or do you have a specific team dedicated to do so?
The truth of the matter is that I do my best to keep up to date with complaints that come in, so I
regularly talk to the support teams to ask how things are going, ask what the student reactions to
upgrades are and I am very conscious of the first few weeks when the upgrade has been
implemented to see whether there is any negative feedback. Truthfully though, there are probably
occasions where I dont get told, but I try my best to keep abreast of whats coming in and I wouldlike to think that if complaints continually came in, which is the reason why you are carrying out this
-
8/13/2019 Report Final Draft 1.3.1
33/52
Page 33of 52
study, and we have got a problem, that I would respond to that and I would use my resources (the
250,000 and the 180,000 for NILE itself) in a different scenario, so I do my best.
Q9) Is there flexibility for new features to be added to NILE? If the plug-in was not available by
blackboard, how would you go about adding one in?
There is yes. The package that blackboard offer us at the moment (the 180,000 has the options to
add plug-ins for additional services. I suppose the down side is that for any new plug-ins that we add,
it would cost us more in terms of an extra renewal fee, but yes, there is a possibility to add more ifyou felt that the product at the moment doesn't offer a good enough service. We don't have staff in
the IT team who are competent enough to recode the product. So if there is a feature that we would
like to include that doesn't come from blackboard then at the moment we would be stuck and that
could lead on to looking at a different product or changing our contract and look for a different
provider such as Moodle or the others out there, so yes we can't make adjustments ourselves, we
rely on blackboard and what they come up with.
Q10) Would you consider reforms of policies associated with NILE that apply to lecturers possibly
making NILE training compulsory so that they can comply with your standards and your legislation
about electronic material being available would that be a possibility?
In the past, we have always tried to be nice by offering the training as support to them, but iflecturers don't take advantage as you know, it could obviously impact on the service. If you could
prove to us, possibly by looking at patterns in satisfaction that training makes a difference to the
student experience, I could possibly take that to the vice chancellor and go about making the
training compulsory. So the answer to your question is yes, but I would need some strong evidence
so the results that you would get to us in a report would be very important.
Q11) The University of Northampton is a business organisation. Would you consider that to
improve NILE you would have to invest money in blackboard as a business? In the market right
now there are open source systems that can be improved, are there facilities in the university that
could do that?
Absolutely, there is already the two sums of money that I have previously mentioned which showthat we are committed. The 250,000 for support and 180,000 for the licence show that we take it
very seriously. If you prove that what we are offering at the moment isn't good enough then I would
press for at the top levels such as the vice chancellor to ask for more money, I think that it is very
important. If you can prove that NILE as it is isn't working then 100% you have my guarantee that I
would take it to the management top level requisites of this institution and tell that that because it
isn't working, we need another solution. So I am open, even though it would be very hard as we
have had a relationship with blackboard for a long time, if you could prove with your investigation
then you will have my commitment to making changes.
Q12) What kind of support documentation is available for the software, and how available is it, so
do the staff and students know whether it is available?I've mentioned the support team that is funded by the 250,000. In that team we have a couple of
learning technologists, and they are responsible for documenting NILE in terms of the features that
we offer to staff and students and providing support material. So if you go onto NILE at the moment,
anywhere that it says or gives you a link to help that support documentation has been created by us.
So absolutely, we want to make the training and support as good as possible, not only do we do the
training during induction for staff and students, where they were invited, but we also produce lots
of online training material and the last piece of material that my learning technologists produced
was a tutorial for students on how to use the blog in NILE, again whether staff and students are
using this information is a different question, but we are absolutely committed to producing so
quality support.
-
8/13/2019 Report Final Draft 1.3.1
34/52
Page 34of 52
Q13) How do you manage usage overload on NILE?
As part of the agreement we have, the 180,000 service agreement, our servers are paid for, and
believe it or not, are based in the Netherlands, and when we paid for the service, we predicted the
amount of traffic that we had and then added an extra 20% capacity, so theoretically, we have
enough server capacity to cater for the existing student body plus spare, so if there was a massive
intake of students, so if 2000 more people came to the university next year, we've still got enough
server capacity to cope with that. It would be extremes that we would have to pay extra money for
to not be enough, and hopefully, I'd like to think that students don't have difficulty logging on andthat availability for NILE has been good, and that regardless of how many people are online, we
don't have any noticeable degradation of performance of the system.
Q14) Is there any functions for areas of NILE which you believe could be added or improved on?
I take the advice of two parties here. Our IT staff who deal with problems when students raise them,
and I have to listen to their recommendations. And I also like to keep abreast of all the information
that comes though from blackboard themselves, so they have a research team and a team that
develop the software, and if they tell us that they could add a new feature to NILE then I listen to
that, but decisions on upgrades and where we go in and how we deal with the product basically
come from staff experience when dealing with students and from NILE blackboard
recommendations.
Q15) In terms of training for both students and staff, is it externally sourced or do you create it?
And if there is a problem, would you be willing to revise the way that it is provided?
Training at the moment was initially supplied by blackboard. Once our IT team were trained,
everything has been serviced by us, and we don't pay blackboard for training. In terms of if it was
found to be inadequate, I would 100% put everything we have into resolving that whether that
would mean making a new policy within the institution about how people are trained and I would
support any recommendations that you make.
Q16) If NILE had a refurbishment, would there be any parts of the website that you would like to
keep that same?The interface for logging on at the moment, we would like to keep. Even if the services would
internally change, we would like to keep the portal the same, and the reason why I say that is
because we have done a lot of work recently in linking up NILE and the sites to our grading system so
tutors are supposed to use the tool within NILE called grade mark and put the grades that have been
achieved by students in there, and the fact that we have linked our portal to our university system
for grading, we wouldn't want to change. So to go back, the one thing that we would need to keep
the same would be the overall infrastructure of the access portal, but we would be prepared to add
new features inside.
Q17) The initial concepts of NILE originated from helping educational organisations in different
aspects such as organising documents in one easily accessible area, when complaints are made,does this hinder the product or does it still succeed?
I really try to take notice of what the student experience is and whether they are happy or not and I
ask my IT staff to log a number of complaints even if they don't tell me the exact details. Over the
last 6 months, remembering that we have around the region of 9000 full time students and another
3-4000 distance learners, we've had complaints about QOS in the region of around 2/3 a month. The
stats are updated regularly and we have peak periods of dissatisfaction what is usually around the
start of the year where people have problems with registration and that it probably goes to about
20, but there are months where there are not complaints at all, so based on that, I feel that the
service is reasonably successful.
-
8/13/2019 Report Final Draft 1.3.1
35/52
Page 35of 52
Q18) Would you tie down the common problems that students get to the training that is provided
to them?
One of the things that we have been trying to do is when common problems materialise, we try to
write help or tutorial aimed at those problems. The numbers that I have quoted earlier are written
complaints. There is a possibility that there is dissatisfaction, but it may just be where people have
complained and IT services have solved the issue and not logged it. So it is a possibility that there are
more complaints that previously quoted, but the just don't go any further than the student phoning
in and getting the issue solved.
Q19) As there are a lot of students on NILE every day, do you ever experience any performance
issues?
We are really happy that the servers back-up system that we have for data such as lectures and
material as we have never lost any data or unable to access it for anything longer than maybe 24hrs
and that is in all the years that we have used the software since the early 2000s. Generally, we are
happy with the service from a technical perspective.
Q20) Are those that have disabilities of a lower priority than those without when they have issues
with the software?
No, that's wrong. The disabled students are actually a high priority because of the legislation that thegovernment has put down. If we were seen to be ignoring people who were disabled, not only would
it give a bad impression for people outside of the institution, but we could be fined and liable to
prosecution. So you will find the opposite. We are actually really concerned about that region. There
are situations where we have done our best to provide a good service, we actually have a blind and
partially poor hearing joining us next year where we have already done the learning
recommendations for their disability, so we do our absolute best to deal with each disabled student
by looking at their needs and trying to make sure that generally, that NILE is compliant with the
legislation and we do our best.
Q21) Are you happy with the service that youve got at the moment?
A: The service can be put into two parts, Blackboard releases updates, and technical informationavailability of service I am happy .In terms of my satisfaction with the way the products being used,
no. I get very frustrated
Q22) I mean exactly with NILE, not with our university. We are paying money for our university to
manage it and for the product (NILE).
A: Yes. Im happy with what weve got at the moment, the value for money that we get. For
180,000 I think it does represent a good figure when you look around the market and you look at
the service we get.
Q23) You said that the login site is important to you, the way it is at the moment. Are there any
other features that you would particularly like to see?A: Yes. The login is absolutely essential because its tied in with student IDs and grades, we couldnt
change that dramatically. In terms of the other features within NILE, I think what weve got isnt bad
but its important Im ready to listen. If you do a study and find that 95% of students think we need
an extra feature and we havent got it then I would be willing to invest extra money. Or alternatively,
looking at another product if it is essential.
Q24) Theres probably people that have problems with NILE and dont make a complaint.
A: I really believe in the training. Like you say, there are possibly people out there that are not
happy, who dont complain and just put up with what theyve got. I would advocate training earlier
off.
-
8/13/2019 Report Final Draft 1.3.1
36/52
Page 36of 52
Q25) Has there been any research like this done since the system was introduced?
A: The research that weve had has been only a collection of the complaints. The project that youre
doing at the moment is the first time weve commissioned software engineers to look into this issue.
Q26) We were told that if people are unsatisfied with the service what is the process to change it?
You are the manager, its your decision to change it. What happened there?
A: As a team with me as the lead weve come up with a policy for effectively service and what the
service includes are announcements, grade marks etc. Theres a written document for what webelieve weve got. If we wanted to change that, the first thing we would need to do is based on your
recommendation wed give a reason for the change. If I wanted to do it then it would go to the top
level management university executive which includes the pro vice chancellor. I would put forward
my case and explain that based on the evidence we need to change this, and then he would either
give me the go ahead, if it required extra funding then that would be on a table to say we need x
number of pounds. He would then say no and we would have to leave it as it was but basically I take
the recommendation forward to the management assuming theyre happy and would make the
change.
Q27) When you chose Blackboard did you choose it for the applications the universities already
had or for the applications blackboard had?A: The infrastructure weve got as a university was a consideration when we took them on-board.
We wanted to include things like compatibility with our existing legacy database for the library and
things like that. What Ive found now though is that Blackboard is usually the standard and then
were changing our technology to comply with that. Initially we were worried about compatibility
but now were more than willing to change.
Item 3- Basic paper draft
Intro:
Basic info:
Gender:
Male [ ] Female [ ]
Which school do you study within?:
Arts [ ] Sciences [ ]
Are you fulltime distance?:
Yes [ ] No [ ]
Are you an international student?:
Yes [ ] No [ ]
Year:
1st [ ] 2nd [ ] 3rd [ ] 4th [ ] Above [ ]
-
8/13/2019 Report Final Draft 1.3.1
37/52
Page 37of 52
Usage:
How often do you use Nile on a weekly basis?:
Multiple times daily [ ] Daily [ ] Multiple times weekly [ ] Less [ ]
Do you find Nile easy to navigate?:
Strongly agree [ ] Agree [ ] Neutral [ ] Disagree [ ] Strongly disagree [ ]
How happy are you with Nile overall?:
Strongly agree [ ] Agree [ ] Neutral [ ] Disagree [ ] Strongly disagree [ ]
Have you ever had any issues with Nile previously?:
Yes [ ] No [ ]
If yes, what sort of issues?:
Did IT support help solve your issue(s)?:
Yes [ ] No [ ]
Features:
Which features do you use, and how do you like them?:
Feature
Like
Dont like
Don't use / Didn't know
-
8/13/2019 Report Final Draft 1.3.1
38/52
Page 38of 52
Would you like to see AB included as a feature of Nile?:
Yes [ ] No [ ]
How satisfied are you with the features of Nile?:
Strongly agree [ ] Agree [ ] Neutral [ ] Disagree [ ] Strongly disagree [ ]
End
Have you got any other comments about Nile?:
Any comments about this survey?:
Ending statement/thanks
Item 4- Complete questionnaire used
The full questionnaire can be found at the following link:
http://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&refer=
http://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&referhttp://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&referhttp://kwiksurveys.com/app/rendersurvey.asp?sid=g3nl352xm2u20yc108222&refer -
8/13/2019 Report Final Draft 1.3.1
39/52
Page 39of 52
Item 5 - Raw results from questionnaire
Total Male 24
Total Female 28
Total 52
First Year 24Second Year 8
Third Year 17
Fourth Year 3
Fifth Year 0
Degree 45
Foundation 2
HND 3
Post-Grad 2
Full-Time 52
Part-Time 0
School of the Arts 15
Northampton Business School 11
School of Education 6
School of Health 4
School of Social Sciences 5
School of Science and Technology 13
Northampton 51
Moulton College 1
Tresham 0
Are you international
Yes 6
No 46
Did you receive training for Nile?Yes 30
No 22
Do you believe this training help?
Yes 23
No 7
Do you believe this training was enough
for you to use NILE well?
Yes 21
No 9
-
8/13/2019 Report Final Draft 1.3.1
40/52
Page 40of 52
How long was the training?
1 Hours 27
2 Hours 3
3 Hours 0
4+ Hours 0
What type of training was it?
Talk from Lecturer 21
Talk from IT specialist 10
Video 1
Web Based 2
How often on average do you use Nile
on a weekly basis?
0 1
1 6
2 4
3 9
4 13
5 18
6 or more 0
What are your thoughts on the
statements below?
Strongly
Agree
Agree Disagree Strongly
Disagree
Overall I am very statistfied with Nile 8 37 6 0
Nile is easy to use 10 34 6 1
I would like to take another course using
NILE
3 15 26 7
Nile helped improve my learning 3 36 9 3
Have you ever had any technical
difficulties with NILE?
Yes 22
No 29
Did you manage to resolve the issue?
Yes 20
No 2
What would/did you use to help resolve
technical difficulties with NILE?
IT help desk 7
NILE help tab 4
A member of staff or another student 10
Other 5
-
8/13/2019 Report Final Draft 1.3.1
41/52
Page 41of 52
Roughly how often on average do you
have problems with NILE?
Never 4
Once a year 8
Once a month 10
Once a day 0More than once a day 0
Have you ever used another virtual
learning enviroment (VLE) before such
as Moodle, Frog or Sakai etc
Yes 17
No 34
Which VLE have you used? (You may
tick more than one answer)Moodle 11
Frog 0
Sakai 0
Boddingtons 0
Other 7
What is your opinion on the statement
below?
Strongly
Agree
Agree Disagree Strongly
Disagree
Nile is easier to use than other VLEs I
have used in the past
3 8 5 1 17
Nile has better functions that other VLEs
I have used in the past
3 10 4 0 17
Nile has helped me more than other
VLEs I have used in the past
4 6 7 0 17
Which features do you like, dislike or
don't use?
Strongly
like
Like Dislike Strongly
Dislike
Don't use
Personalized pages 4 31 3 0 13
Email notifications 11 27 6 1 6
Module annoucements 15 29 6 0 1
Calendar 3 25 3 1 19
SmartPhone app 6 24 1 2 18
Nile search engine 3 23 5 0 20
Discussion boards 2 30 4 1 14
Course blogs 2 26 3 0 20
Course wikis 2 16 5 1 27
Which features would you like to see
added to Nile?
Would
strongly
like added
to NILE
Would
like
added to
NILE
Would
not like
added to
NILE
Would
strongly
not like
added to
NILE
Wouldnt
Use
-
8/13/2019 Report Final Draft 1.3.1
42/52
Page 42of 52
A section dedicated to group work 22 22 2 0 5
Peer feedback tool 8 34 4 1 4
Social networking 10 21 8 1 11
Survey tools 13 22 8 0 8
Course blogs 9 29 3 0 10
Instant messaging 14 20 6 2 9
Course file search 19 27 3 0 2
Voice tools 8 10 14 1 18
Course glossary 10 30 5 1 5
Journals 14 21 5 3 8
Podcast 10 19 8 0 14
Question creation 17 23 6 0 5
Detailed user profiles 8 19 9 1 14
Item 6Presentation
The following are images of the presentation to the management team, but due to the animations
certain slides have elements which overlap as such these images are not representative of the
presentation.
-
8/13/2019 Report Final Draft 1.3.1
43/52
Page 43of 52
-
8/13/2019 Report Final Draft 1.3.1
44/52
Page 44of 52
-
8/13/2019 Report Final Draft 1.3.1
45/52
Page 45of 52
-
8/13/2019 Report Final Draft 1.3.1
46/52
Page 46of 52
-
8/13/2019 Report Final Draft 1.3.1
47/52
Page 47of 52
-
8/13/2019 Report Final Draft 1.3.1
48/52
Page 48of 52
-
8/13/2019 Report Final Draft 1.3.1
49/52
Page 49of 52
Item 7 - Presentation transcript
-
8/13/2019 Report Final Draft 1.3.1
50/52
Page 50of 52
Hello and welcome to team 2s presentation!
[Next slide]
The team was hired to investigate the Northampton Integrated Learning Environment to ensure it
meets a required standard. This presentation shows the results and conclusions of this investigation.
[Click]
Before we start let us quickly explain why you should care. Our investigation found that there is a
direct correlation between the average number of problems a student has compared with
satisfaction and if they thought NILE helped their learning. Based on this information it is likely that ifNILE is unsuitable or not the best product on the market then NILE is damaging the University's
reputation and costing you money.
[Next slide]
We started out the investigation by creating aims and objectives to keep the research focused.
[Click]
And then we created a list of objectives required for us to fully access and evaluate this aim.
After we had identified the problem domains we began investigating three areas; Starting with the
current product specification
[Click]
Here we have a list of NILEs features, but we also investigated the Embedded tools which includedthe HTML content editor, Help and support mechanisms mostly provided by the University which
include PDFs and videos, and finally NILEs Software Expansion Potential where add-ons can be added
to customise NILE.
[Next slide]
The second problem domain was the comparable systems
[Click]
We initially found three popular virtual learning environments
[Click]
Then by researching there features and comparing them to NILE we created this features table
which has been provided in your hand out.
The final problem domain investigated was the IT Support Help Desk Staff. We did this by
interviewing the IT manager
[Click]
Here we have the most important statements he provided us with.
Using the problem domain review, a list of questions to ask students was created and a basic paper
based questionnaire was made. As online survey would improve efficiency by increasing distribution
and analysis, potential online survey tools were investigated.
[Click]
KwikSurvey was chosen as the survey tool to use as it allowed unlimited questions unlike other tools
researched into.[Next slide]
After internal testing of the questionnaire had finished, we used a small sample of students to
confirm the questionnaire effectively gathered feedback. To do this we asked them to complete the
survey and then asked them several basic questions about the questionnaire.
[Click]
The sample consisted of six students and only one had any significant feedback, which is as shown,
one was about the text explaining the reasoning of the question the other was about an option for
one of the questions.
[Click]
Using this feedback the questionnaire was improved and was then ready for complete distribution.
[Next slide]To distribute the questionnaire we chose high foot traffic areas in each campus and moved around
-
8/13/2019 Report Final Draft 1.3.1
51/52
Page 51of 52
these areas using laptops and tablets to get the largest range of feedback possible. To increase
student participation we used incentives to students who completed the survey.
The next relation we looked into was between training hours and average number of problems a
year, although there is an increase in problems going from no trainings hours to one, we believe this
is explained because students use a higher number of features.
[Click]
Here we investigate training hours against several other areas such as satisfaction. Thequestionnaire asked how strongly they agreed with several statements and this was converted into
weighted averages on a scale where 2 is strongly agree, 1 is agree 0 is neutral, -1 is disagree and -2 is
strongly disagree.
As you can see there is a significantly higher satisfaction, ease of use and improved learning with
higher training hours.
[Click]
Here we have created a prediction for three training hours using a logarithm regression line. As you
can see while there is an increase, the benefit is small and will likely be reduced further by
diminishing returns.
Because of this information, we believe the first change should be to make training two hours as a
minimum as one training hours is insufficient and three is to large[Next slide]
As shown by these graphs the most effective training is is video training as it has the lowest number
of problems and shares the highest satisfaction. This would suggest the most effective and efficient
way would be to use the existing video tutorials to create a complete video tutorial which students
would watch during training and that would teach them all features.
[Click]
As shown by this graph, the school of health had a much larger average number of problems
compared with other schools, in addition to the video, this school would greatly benefit with extra
support such as n IT specialist.
Five features had over 35% not used, although some features are likely not used due to personalpreference, most are likely not used because staff do not promote those features therefore students
do not use them. Together with the IT managers stating that certain members of staff do not accept
training, we believe training should be made compulsory for staff members to ensure they utilize
NILE to its full potential. In addition to this, the IT managers stated that the updates to NILE where
every 6 months, as such to ensure staff are knowledgeable about new features, small training
sessions for staff should employed after every large update.
[Next slide]
The results show a high correlation between year of study and average number of problems and no
correlation has been found between weekly usage of Nile and number of problems as well as year of
study and satisfaction. This would suggest that training is required for all levels as most problems are
not resolved easily with experience, but are not flaws with NILE because 90% of all problems wereresolved. Training twice a year and for all years will allow users to stay up-to-date for new features
and features more suited for later years.
As the NILE usage increases so does satisfaction and the number of problems, this indicates that
NILE has good features which users want but these features have not been fully trained and so
problems occur. This would indicate that more features are not required but could further increase
satisfaction and performance. And as 76% of students who completed the survey agreed or strongly
agreed that NILE helped their learning NILE is mostly suitable for the University of Northampton.
[Click]
Only three features are widely wanted by users; a section dedicated to group work, a que