the impact of job aids on the transfer of learning in the
Post on 20-Apr-2022
3 Views
Preview:
TRANSCRIPT
Running head: THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 1
The Impact of Job Aids on the Transfer of Learning in the Workplace
Dean Stubbs
James Madison University
Fall 2019
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 2
Abstract
This applied research project examined the use and effectiveness of just-in-time learning, adult
learning theory, and job aids on the transfer of learning in the workplace. Design considerations
were also reviewed to consider effectiveness. A review of the literature was conducted and
uncovered a connection among just-in-time learning, adult learning theory, and job aids on the
transfer of learning in the workplace. However, there are no studies that look exclusively at the
use of web-based systems. Leveraging this type of training can be more effective for learners,
cut down on the amount of initial system training, and reduce the amount of follow-up training.
This study will be conducted in one higher education settings. Future studies in wider range of
workplaces will likely contribute more generalizable results.
Keywords: job aids, transfer of learning, adult learning theory, workplace
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 3
Chapter One
Introduction
James Madison University (JMU) uses a web-based procurement system (eProcurement
system) to purchase goods and services. Electronic Virginia (eVA) is the name of our
eProcurement system, and all agencies in The Commonwealth of Virginia are legally required to
use this system. JMU has approximately five hundred active system users at any given time and
we receive about four to ten new user access requests each month. System users range from
administrative assistants to department heads, vice provosts, and assistant vice presidents.
In my role at JMU, I serve as the entity primary security officer and lead support for
eVA. This means I’m responsible for creating and deactivating user accounts, responding to eVA
helpdesk calls and emails, among other activities. I interact with a large subset of our userbase in
a given month for a variety of questions, which can be broken down into two overarching
categories: procurement/purchasing policy and system functionality. These questions come from
new and seasoned system users alike as changes to eVA and procurement policy are
implemented.
Current Training Model
Before a faculty or staff member can be granted system access, they are required to attend
our eVA Overview training session. The training was originally created in 2003, and was
designed using Behaviorism as the primary theoretical framework. Although this training has
been running at JMU for over 15 years, the overall format has not changed, while the content has
been updated over the years to adjust for system changes. This training is three and one-half
hours long and has two main sections. First, learners spend an hour in a lecture format learning
basic procurement and purchasing policies that will guide their work. Some of these items
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 4
include: mandatory sources, the importance of our supplier diversity program (SWaM), restricted
items, when the procurement office needs to be involved in a purchase, and things that are
considered illegal. Then, for the next two and one-half hours, the leaners sign into the training
version (training instance) of the eVA system, which is identical to the real system but orders
created in the training instance are not transmitted to vendors. During the second half, learners
follow hands-on demonstrations and exercises. Each demonstration covers real-life situations
that the learners may encounter in their departments from within the Ariba application, which is
the purchasing module of eVA. This can be visualized in the eVA Overview organization map,
Figure 1.
Figure 1. eVA Overview organization map
There are three main performance outcomes for the training. First, the learners will have,
at a minimum, a base level understanding of the system that will allow them to use the eVA
system with minimal difficulty. Second, the learners will have a higher-level understanding of
appropriate procurement policy. Finally, the learners will know the different resources and
eVA Ovierview
Policies
Lecture Based
1 Hour
• Accepatble JMU Purchasing
• Supplier Diversity (SWaM)
• Restricted Items
• Mandatory Sources
eMall Usage & Functinality
Hands-On / Demonstration Based
2.5 Hours
• Catalog/Punchout Orders
• Non-Catalog Orders
• Confirming Orders
• Ad-Hoc Orders
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 5
contacts they may utilize if they have system or policy questions and issues. However, there is
no formal assessment to gauge learners’ comprehension of the content at the conclusion of the
training, nor transfer to their workplace. Rather, as long as the learners have completed the
training and submitted access request paperwork, they are given a user account to the eVA
system.
Although I am the owner of the eVA Overview class, we have the IT Training group at
JMU facilitate the training, and I attend to serve as the subject matter expert (SME) and answer
various questions that may come up from users that the trainer may not be able to answer. At the
start of the training, we have the learners go around the room and share their name, what
department they are from, how long they have been at JMU, and what goods and services they
will be purchasing in eVA. Typically, those in the training are either brand new to the university
and as a result, are taking a variety of classes in a short period of time to get access to the many
systems they need for their job role. This often causes overload and confusion due to the high
number of new terms and varying systems. However, a smaller subset of individuals come to the
training after being at JMU for a longer period of time and aren’t having to complete four or five
system training courses in a short period of time. Further, those who have been at JMU longer
already have a rough understanding of procurement and purchasing policies, and how those
interact with other areas: finance and information technology.
Future Of eVA
The contract that the Commonwealth of Virginia utilizes for the whole eVA system is set
to expire in June of 2021, as a result there is an ongoing procurement to solicit for a new
eProcurement system to replace the existing one. The impact of this change will be significate,
not only to the user base at JMU, but across all agencies in Virginia. Based on the current
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 6
timeline available, the new contract will be awarded around August 2020, and then the
implementation process will begin. All current users must be in the new eProcurement system
no later than June 30, 2021, but many agencies will transition to the new system in late spring
2021. While the branding of eVA will remain the same, the eProcurement system behind it will
completely change and, given that the current software has been around for over 20 years, any
new system will have a dramatically different user interface and functionality. As such,
implementing the new system will require substantial retraining not only at JMU, but for the over
11,000 system users across the Commonwealth. The timing for this research is very fortunate in
that the findings can help guide the conversation and design of the new system training for
effective retention. For JMU, we will likely develop our own in-house training materials and
offerings. However, the group that manages eVA is also aware of the research I am conducting
and has asked for the findings at the conclusion.
Summary
Given how prevalent informal learning, such as just-in-time learning, is in the workplace,
this review is being undertaken to gain insight into what studies about job aids in the workplace
have already been conducted and the results of those studies. The main goal of this applied
research project is to answer these research questions:
1. How do learners perceive traditional eVA training?
2. How do learners perceive the transfer of the eVA training?
3. How did the job aid assist the learner with the transfer and using eVA in their workplace?
4. What components of training are most preferred and least preferred for workplace
training?
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 7
Being able to answer these questions will better inform and guide future training at JMU for
faculty and staff that use the eVA system. As well, the results of this study will play an important
part in helping guide the conversation and the design of the retraining program not only at JMU
but at agencies all across The Commonwealth of Virginia.
Chapter Two
Overview
Around the world, companies heavily invest in training for their employees. For example,
in 2018, companies located in the United States spent $87.6 billion on training and development
(2018 Training Industry Report, 2018). A vast majority, an estimated 80 percent, of workplace
learning that takes place, happens informally (Cerasoli et al., 2018). However, many
organizations spend less than 20 percent of their budget to financially support alternative
learning or training methods, such as just-in-time learning (Cerasoli et al.). An even smaller 10
percent of learning from training is successfully transferred back to the workplace (Heilmann,
Bartczak, Hobbs, & Leach, 2013). Given this disparity between supporting alternative learning
or training methods and their prevalence, a review of workplace training and Just-In-Time
Learning in the workplace is needed to better understand their impacts on employee learning,
workplace readiness, and the transfer of learning. The use of job aids to help improve the transfer
of learning in the workplace is supported by several components: transfer of learning, just-in-
time learning, adult learning theory, workplace training, and with the tools of Education
Technology. Figure 2 models this overarching theoretical and conceptual framework.
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 8
Figure 2. Theoretical and conceptual framework
Definitions
Below is a list of relevant terms that will be used in this paper, along with their
definitions.
Just-In-Time Learning (JITL), is made up of several attributes: learner-led, takes place
outside of a formal learning environment and is made up of a focused lesson that is considered to
be a form of informal learning (Boileau, 2018). JITL content is targeted at a specific need and is
used just prior or while a learner is attempting a given task (Davis, Garcia, Wyckoff, Alsafran,
Graygo, Withum, & Schulman, 2012). Several benefits that JITL provides are reduced
disruptions because learners are not required to leave their office for training, and the minimized
impact on workers' daily schedules (Boileau).
Transfer of
Learning
Workplace Training
Just-In-Time
Learning
Adult Learning Theory
Educational Technology (Job Aids)
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 9
Job Aids are a type of performance support tool, which are used as an instructional
intervention (Spaulding & Dwyer, 2001). Although job aids vary in complexity and type
(procedural, informational, & decision-making), their goal is still to help scaffold learners
through various tasks on the job (Agola & Stefaniak, 2017). These tools are considered a form of
just-in-time learning and can be digital or paper-based (Florez-Arango, Iyengar, Dunn, & Zhang,
2011).
Transfer of Learning occurs when learning in a given context enhances performance in
another context (Perkins & Salomon, 1992). A transfer is more likely to take place with near
transfer, when the work task is highly similar to the learning (Blume, Ford, Baldwin, & Huang,
2010). One way to help increase the likelihood of the transfer of learning is for learning to be
authentic and take place in the performance context (Boileau, 2018).
The field of Educational Technology has changed its name and definition five times
over the last 50 years. In 2012, the Definition and Terminology Committee of the Associate for
Educational Communications and Technology updated the definition of the field as follows:
“Educational technology is the study and ethical practice of facilitating learning and improving
performance by creating, using and managing appropriate technological processes and resources
(Januszewski & Molenda, 2013, p. 1).
Adult Learning theory, also known as andragogy, was developed by Malcolm Knowles
in the 1960s, and updated in the 1990s, to better understand how adults learn (Woodard, 2007).
Andragogy is comprised of five assumptions about adult learners: self-concept – adult prefer a
self-directed approach, learner experience – adults often have prior knowledge to build off of,
readiness to learn – adults are ready and want to learn if there is a reason to do so, orientation of
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 10
learning – adults want the learning to be practical and relevant to their work, and motivation to
learn – adults want to learn because of intrinsic factors (Malik, 2016).
Eligibility Criteria & Information Sources/Search
For the review of professional literature, only current literature, articles published
between 2006 and 2019, in the English language, and from peer-reviewed academic journals that
were available as full-text online were considered. In addition, articles needed to have an adult
learner focus, either students in higher education or adults in the workplace. The literature review
on this topic was based on searches using the EBSCOhost Research Platform of publications
within the research databases Education Resources Information Center (ERIC) and Business
Source Complete. The main goal of this review was to see what professional research has been
done about job aids, and any connections to workplace training, transfer of learning, just-in-time
learning, educational technology, and adult learning theory. In addition to looking at the transfer
of learning in traditional training. For research questions one and two, the search terms:
“classroom AND transfer of learning,” were used, and for research questions three and four: “job
aids OR performance support aids”, were used.
Method of Summary
Each article was reviewed and grouped with the research question it most relevantly
addressed. Then a systematic review of each article was done to gain insight into methods,
subjects/participants, settings, results, and conclusions. Finally, a high-level review of each study
and its possible transferability to the research questions was done.
Findings
Theoretical & conceptual framework.
Workplace training is a vital part of any organization, this is especially true as new
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 11
employees need to be brought up to speed on a variety of systems, processes, and policies.
Woodard (2007) found that organizations could still ensure employees learned need content for
their job while allowing the employees to be self-directed and learn content as it was needed; all
while increasing the learners’ perceived readiness to do their job effectively. Just-in-time
learning is one method through which many aspects of adult learning theory can be leveraged for
training in the workplace (Hague & Logan, 2009). In fact, Ley, Cook, Dennerlein, Kravcik,
Kunzmann, Pata, & Trattner (2014) found that just-in-time learning can be highly effective, but
that training professionals often struggled to support it due to scalability and the often-formal
nature of many learning management systems. One instructional intervention that can help
bridge this gap is job aids; because of their versatility in complexity, type, and format, they can
be utilized in informal ways (Agola & Stefaniak, 2017). Finally, an important aspect
underpinning all of these concepts is the extent to which the transfer of learning is facilitated and
can be improved through the use of Adult Learning Theory, just-in-time learning, and job aids
(Blume, Ford, Baldwin, & Huang, 2010; Boileau, 2018).
Traditional training & transfer.
Benefits.
Classroom training that is led by an instructor has been a popular method for delivering
training in organizations. One study found that 92 percent of employers reported using
classroom-based training, which was led by on-the-job training at 94 percent (Saks & Burke,
2014). This popularity is due to the various benefits of this method, students are able to receive
real-time feedback from the instructor and because students are away from their normal work
environment they are less likely to encounter daily distraction while learning (Chatzimouratidis,
Theotokas, & Lagoudis, 2012).
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 12
Criticisms.
While classroom training is still heavily used in the workplace, there are a variety of
issues with this training method. Classroom training can suffer if the instructor is not fully
engaged with the topic or if the instructor doesn’t have a good understanding of the topic
(Chatzimouratidis, Theotokas, & Lagoudis, 2012). In a study by Saks and Burke (2014) that
looked at the transfer of learning, they found that classroom training didn’t have a significant
positive correlation with the transfer of learning, but on-the-job training activities did. Other
research suggests part of this may have been caused by the learning taking place in the
performance context or an authentic learning environment, rather than a classroom that doesn’t
match the learner’s work setting (Herrington, Reeves, & Oliver, 2014). Speicher and Kehrhahn
(2009) found that transfer from the classroom setting into the job setting was also hindered by
only using one case study, rather than having multiple case studies with variations for learners to
work through. Finally, classroom training makes it difficult to fully utilize concepts from Adult
Learning Theory, such as self-direction, orientation to learning, and motivation (Woodard,
2007). This is caused by not allowing the learning to self-direct the learning experience and the
lack of experience with real situations that motivate learners to go learn and search for answers
(Woodard).
Data gathering.
A vast majority of the data gathering was conducted through surveys. When analyzing
the data from the surveys, pairwise comparison, multiple regression analyses, or t-test analysis
were used. (Chatzimouratidis, Theotokas, & Lagoudis, 2012; Saks & Burke, 2014; Speicher &
Kehrhahn, 2009). Saks and Burke research used training and development professions as their
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 13
survey participants. Another two studies surveyed employees directly about their perceptions of
various training types or processes (Chatzimouratidis, Theotokas, & Lagoudis; Woodard, 2007).
Job aids & transfer.
Benefits.
One strategy used to help complement existing training and increase the likelihood of
transfer is the use of job aids (Baldwin‐Evans, 2006). However, job aids may be used
independently of any formal training (Florez-Arango, Iyengar, Dunn, & Zhang, 2011). These
job aids are used by employees while they are trying to complete an on the job task or
assignment; this is beneficial due to the learning taking place in the performance context for
increased transfer and because it allows for self-directed learning (Agola & Stefaniak, 2017;
Malik, 2016). When job aids are used just prior to or while the learner is completing a task the
learner has a minimized impact on their daily work schedule and less lost productivity (Boileau,
2018). Because adults want learning to be relevant to their work, having adult learners utilize
job aids, allows the learner to see the direct practical and relevant use of the job aid (Woodard,
2007). This also means the learner is scaffolded through the job task (Agola & Stefaniak).
Finally, when good educational technology design principals and tools are utilized, job aids can
increase job performance (Harvey, Jennings, Chinyama, Masaninga, Mulholland, & Bell, 2008).
Criticisms.
Although job aids are beneficial, the time required to effectively design and create them
means they may not be effective for all tasks and situations (Burke & Hutchins, 2008). Rather,
thoughtful consideration must be given to ensure an appropriate instructional intervention is
used. If a leaner has a difficult time accessing a job aid, whether in print or digital, or finding
information within the job aid it can have a negative impact on learning (McManus & Rossett,
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 14
2006). Lancaster, Di Milia, and Cameron (2013) completed a study and found that culture,
policies and a lack of encouragement from supervisors played a role in the lack of transfer of
learning in the workplace. As a result, there may be other considerations that need to be looked
at when attempting to increase transfer. However, these considerations are not being looked at in
this research as these factors fall into managerial and human resources policies, which are out of
scope.
Data gathering.
Researchers used a mix of data gathering methods to look at the impact of job aids:
surveys, observations, and interviews. Several studies made use of qualitative
NVivo coding or quantitative content analysis procedures, for data collected through interviews
or surveys (Burke & Hutchins, 2008; Lancaster, Di Milia, & Cameron, 2013; Malik, 2016).
However, most used analytical software to utilize Chi-square of Independence, Paired t-tests, or
mixed linear mode (Agola & Stefaniak, 2017; Florez-Arango, Iyengar, Dunn, & Zhang, 2011;
Harvey, Jennings, Chinyama, Masaninga, Mulholland, & Bell, 2008). Finally, only two studies
used training and development professions as their survey participants, rather than surveying
workers directly (Burke & Hutchins; Malik).
Summary & Research Gap
There are clear limitations to traditional classroom-based training, which hurt the overall
transfer of learning to the performance context. Overall, job aids are an effective way of
increasing the likelihood of transfer and provide other benefits to the learner. However, job aids
are not appropriate for all situations, as other instructional interventions may work better. There
are also many factors that impact the transfer of learning in the workplace beyond the actual
instructional intervention, such as lack of encouragement for survivors and work culture. Most
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 15
data for this type of research is gathered through surveys, and to a smaller extent observations
and interviews.
While most of the research reviewed looks at transfer of learning with employees in the
work setting, none of the studies looked specifically at the transfer of learning in reference to
learning how to use a web-based system. Rather, studies looked at the transfer of learning in
general for students in a class, new employees, or the perception of effective training from staff
that work in training and development. As a result, there is a need to research what impacts job
aids would have on the transfer of learning with the use of a web-based system.
Chapter Three
Research Design
The purpose of this study is to explore factors that impact the transfer of training in the
workplace and the effects job aids have on increasing transfer. A mixed-methods approach has
been selected so that both quantitative and qualitative data can be collected through two surveys.
The goal of this study is to answer these research questions:
1. How do learners perceive traditional eVA training?
2. How do learners perceive the transfer of the eVA training?
3. How did the job aid assist the learner with the transfer and using eVA in their workplace?
4. What components of training are most preferred and least preferred for workplace
training?
Instruments
For this study, two mixed-methods surveys will be used (Appendix A & B), both of
which were developed by the researcher. The first survey (S1) consists of six questions, four
quantitative and two qualitative. At the end of the first survey participants will be randomly
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 16
presented, using Qualtrics’ Randomizer tool with evenly present elements selected, either a link
to the digital job aid or a link to the printable job aid. In addition, participants will have
instructions to use the job aids over the next week as needed. Approximately one week later, a
second survey (S2) will be sent out to the same participants. This survey consists of seven
questions, six of which are quantitative. A more in-depth overview of both surveys can be found
in Table 1. Both surveys will be located in Qualtrics, along with the data from the surveys.
Table 1
Surveys - Method of Analysis and Data Type per Question
Survey
Question
Number
Survey Question Research
Question
(RQ)
Type of Data Type of
Analysis
S1 – Q1 Approximately how long ago did
you complete the eVA Overview
Course?
Demographics Quantitative Descriptive
Statistics
(Median, Mode,
Mean) S1 – Q2 What is your age? Demographics Quantitative Descriptive
Statistics
(Median, Mode,
Mean) S1 – Q3 What characteristics of the eVA
Overview training did you perceive
to be most effective?
RQ1 Qualitative Thematic
Analysis
S1 – Q4 What characteristics of the eVA
Overview training did you perceive
to be least effective?
RQ1 Qualitative Thematic
Analysis
S1 – Q5 How often do you use eVA? RQ2 Quantitative Descriptive
Statistics
(Median, Mode,
Mean, Standard
Deviation) S1 – Q6 How useful did you perceive the
eVA Overview training to be once
you attempted to use eVA back in
your office?
RQ2 Quantitative Descriptive
Statistics
(Median, Mode,
Mean, Standard
Deviation)
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 17
S2 – Q1 Approximately how long ago did
you complete the eVA Overview
Course?
Demographics Quantitative Descriptive
Statistics
(Median, Mode,
Mean) S2 – Q2 What is your age? Demographics Quantitative Descriptive
Statistics
(Median, Mode,
Mean) S2 – Q3 Which job aid did you use? RQ3 Quantitative Descriptive
Statistics
(Median, Mode) S2 – Q4 How frequently did you use the job
aid?
RQ3 Quantitative Descriptive
Statistics
(Median, Mode,
Mean, Standard
Deviation) S2 – Q5 Did the job aid assist you with using
eVA in your workplace?
RQ3 Quantitative Descriptive
Statistics
(Median, Mode,
Mean, Standard
Deviation) S2 – Q6 How did the job aid assist you with
the transfer of the eVA Overview
training into using eVA in your
workplace?
RQ3 Qualitative Thematic
Analysis
S2 – Q7 Rank these components of training
from most preferred to least
preferred for the type of training you
get at work.
RQ4 Quantitative Descriptive
Statistics
(Median, Mode,
Mean, Standard
Deviation)
Sample
Possible participants for this study will be selected through purposeful sampling. To be
eligible participants must meet the following criteria:
1. Must have taken the eVA Overview Course (IT502) within six months of the
survey (August 2019 - January 2020)
2. Must have an active eVA account
The email addresses of participants that meet the eligibility criteria will be identified
through a listing contained within Procurement Services and compared to the listing of active
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 18
eVA users available through the User Management module of the eVA system, which is only
accessible by the entity’s primary and backup security officers. A final listing of those that meet
the criteria will be created and used by the researcher. An estimated 20 to 50 individuals will
meet the eligibility criteria.
Recruitment
In February of 2020, an email will be sent to all individuals who meet the eligibility
criteria from the researcher’s @jmu.edu email account with the recipients listed in the blind
carbon copy field to ensure only the researcher is aware of the potential participants. The first
email (Appendix C) will contain informed consent and then, at the bottom, a hyperlink to the
survey. The same sample of participants will be emailed again one week later with a link to the
second survey.
Methods
This mixed-methods study will be used to investigate factors that impact the transfer of
training in the workplace and the effects job aids have on increasing transfer. This will be
accomplished through the use of two surveys (Appendix A & B), which will be sent to
individuals selected through purposeful sampling, after IRB approval. Once IRB approval is
received, an email (Appendix C), which includes informed consent, will be sent in February to
the participants that meet the eligibility criteria, which will include a link to survey one
(Appendix A). Approximately one week later, the first survey will close and then the same
participants will receive another email with the link to the second survey (Appendix B). Then,
after one week, the second survey will close. Once both surveys are closed, the data will be
reviewed, in Qualtrics and locally through downloaded CSV / Excel files out of Qualtrics on to
an encrypted and JMU managed computer. However, any incomplete surveys will not be
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 19
included in the data set, and the total number of incomplete surveys will be noted in the final
report. Finally, the researcher will then work through the data analysis process as outlined in the
Data Analysis section.
Protection and security of data.
For both surveys, Personally Identifiable Information (PII) of the participants will not be
collected, and the Anonymize Response option within Qualtrics will be used. Only the researcher
and the faculty advisor will have access to the survey data contained within Qualtrics. The
Qualtrics account of the researcher is password protected and has Two Factor Authentification
enabled. In addition, Qualtrics uses HyperText Transfer Protocol Secure (HTTPS) encryption
for all transmitted data.
The raw data will be reviewed using both a MacBook Pro and Dell desktop, both of which
have an encrypted SSD (solid-state drive). These computers are managed by JMU’s IT
Department; both computers require a password before they can be accessed, so the data on these
computers can only be accessed by the researcher. In addition, both of these computers are
located in the Wine-Price building at JMU, and are located in a room that is locked when not
occupied by the researcher. At the conclusion of the research and defense process, all raw data
and responses to both surveys will be destroyed.
Timeline.
The study will commence in January of 2020 and conclude in April of 2020. For this
study, both the researcher and the participants will be located in Harrisonburg, Virginia on
James Madison University’s campus. Figure 3 contains an overview of this timeline.
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 20
Figure 3. Timeline overview
Data Analysis
After both surveys have closed, the data will be aggregated. The quantitative questions
will be analyzed using descriptive statistics, a total of ten questions, and the qualitative questions
will be analyzed using thematic analysis, a total of three questions. A more in-depth overview of
the analysis types for both surveys can be found in Table 1. With the descriptive statistics
analysis, the researcher will look at the median, mode, mean, and standard deviation. Finally, for
the thematic analysis, the researcher will look for alignment along what learners perceive to be
most effective and least effective to their ranking of components of training, as well as any other
themes and patterns. Dr. Cheri Beverly will serve as the second-rater and independently assign
themes, to ensure reliability among the themes and patterns assigned by the researcher.
Conclusion & Limitations
There are several factors that may impact the overall study. Given the researcher’s role
with eVA and connection to the participant pool, participants could have an unintended bias in
their responses. The instruments being used for the study were created by the researcher and as a
result, the validity of those instruments has not been determined. Finally, because the current
training model does not contain any form of assessment, a smaller time frame of six months was
selected in an effort to increase the likelihood that participants would have recall of the training.
January 2020
•IRB Approval
•Identification of eligible participants
Febuary 2020
•Email eligible participants -survey one
•Email eligible participants -survey two
March 2020
•Analysis of survey data
April 2020
•Compile analysis and complete applied research paper
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 21
This means a limited pool of 20 to 50 participants will be eligible to receive the surveys.
However, the data from this study, at its conclusion, will be able to provide insight into
workplace training and to answer the research questions from Chapter 1.
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 22
References
2018 Training Industry Report. (2018). Training, 55(6), 18–31.
Agola, E. A., & Stefaniak, J. E. (2017). An Investigation into the Effect of Job-Aid Design on
Customer Troubleshooting Performance. Performance Improvement Quarterly, 30(2),
93–120. https://doi.org/10.1002/piq.21245
Baldwin‐Evans, K. (2006). Key steps to implementing a successful blended learning strategy.
Industrial and Commercial Training.
Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of training: A meta-
analytic review. Journal of Management, 36(4), 1065-1105.
Boileau, T. (2018). Informal Learning. In R. West (Ed.), Foundations of Learning and
Instructional Design Technology (1st ed.). Retrieved from:
https://lidtfoundations.pressbooks.com/.
Burke, L. A., & Hutchins, H. M. (2008). A study of best practices in training transfer and
proposed model of transfer. Human Resource Development Quarterly, 19(2), 107-128.
Cerasoli, C. P., Alliger, G. M., Donsbach, J. S., Mathieu, J. E., Tannenbaum, S. I., & Orvis, K.
A. (2018). Antecedents and Outcomes of Informal Learning Behaviors: a Meta-Analysis.
Journal of Business & Psychology, 33(2), 203–230. https://doi.org/10.1007/s10869-017-
9492-y
Chatzimouratidis, A., Theotokas, I., & Lagoudis, I. (2012). Decision support systems for human
resource training and development. International Journal of Human Resource
Management, 23(4), 662–693. https://doi.org/10.1080/09585192.2011.561235
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 23
Davis, J. S., Garcia, G. D., Wyckoff, M. M., Alsafran, S., Graygo, J. M., Withum, K. F., &
Schulman, C. I. (2012). Use of mobile learning module improves skills in chest tube
insertion. Journal of Surgical Research, 177(1), 21-26.
Florez-Arango, J. F., Iyengar, M. S., Dunn, K., & Zhang, J. (2011). Performance factors of
mobile rich media job aids for community health workers. Journal of the American
Medical Informatics Association, 18(2), 131-137.
Hague, C., & Logan, A. (2009). A review of the current landscape of adult informal learning
using digital technologies. Educational Research.
Harvey, S. A., Jennings, L., Chinyama, M., Masaninga, F., Mulholland, K., & Bell, D. R. (2008).
Improving community health worker use of malaria rapid diagnostic tests in Zambia:
package instructions, job aid and job aid-plus-training. Malaria Journal, 7(1), 160.
Heilmann, S. G., Bartczak, S. E., Hobbs, S. E., & Leach, S. E. (2013). Assessing Influences on
Perceived Training Transfer: If I Only Knew Then What I Need to Know Now. Journal
of Business & Educational Leadership, 4(1), 34–48.
Herrington, J., Reeves, T. C., & Oliver, R. (2014). Authentic learning environments.
Handbook of Research on Educational Communications and Technology (pp. 401-412).
Springer, New York, NY.
Huang, Y., Strawderman, L., & Murray, D. (2012). Investigating the Impact of Task Change
Type and Transparency on Transfer of Learning. International Journal of Human-
Computer Interaction, 28(1), 61–71. https://doi.org/10.1080/10447318.2011.568894
Januszewski, A., & Molenda, M. (Eds.). (2013). Educational technology: A definition with
commentary. Routledge.
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 24
Lancaster, S., Di Milia, L., & Cameron, R. (2013). Supervisor Behaviours that Facilitate
Training Transfer. Journal of Workplace Learning, 25(1), 6–22.
Ley, T., Cook, J., Dennerlein, S., Kravcik, M., Kunzmann, C., Pata, K., Trattner, C. (2014).
Scaling Informal Learning at the Workplace: A Model and Four Designs from a Large-
Scale Design-Based Research Effort. British Journal of Educational Technology, 45(6),
1036–1048.
Malik, M. (2016). Assessment of a Professional Development Program on Adult Learning
Theory. Portal: Libraries & the Academy, 16(1), 47–70.
https://doi.org/10.1353/pla.2016.0007
McManus, P., & Rossett, A. (2006). Performance Support Tools: Delivering Value when and
where It Is Needed. Performance Improvement, 45(2), 8–16.
Perkins, D. N., & Salomon, G. (1992). Transfer of learning. International Encyclopedia of
Education, 2, 6452-6457.
Saks, A. M., & Burke, S. L. A. (2014). Is transfer of training related to firm performance?
International Journal of Training & Development, 18(2), 104–115.
https://doi.org/10.1111/ijtd.12029
Spaulding, K., & Dwyer, F. (2001). The Effect of Time-on-Task When Using Job Aids As an
Instructional Strategy. International Journal of Instructional Media, 28(4), 437–447.
Speicher, T., & Kehrhahn, M. (2009). Analogical Reasoning: A Process for Fostering Learning
Transfer from the Classroom to Clinical Practice. International Forum of Teaching &
Studies, 5(2), 52–58.
Woodard, C. A. (2007). Using Adult Learning Theory for New-Hire Training. Journal of Adult
Education, 36(1), 44-47.
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 25
Appendix A
Survey One
Approximately how long ago did you complete the eVA Overview Course?
• One Month
• Two Months
• Three Months
• Four Months
• Five Months
• Six Months
What is your age?
• 18 - 29
• 30 - 39
• 40 - 49
• 50 - 59
• 60 - 69
• 70+
What characteristics of the eVA Overview training did you perceive to be most effective?
What characteristics of the eVA Overview training did you perceive to be least effective?
How often do you use eVA?
• Daily
• Multiple times a week
• Once a week
• Several times a month
• Once a month
• Every few months
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 26
• Never
How useful did you perceive the eVA Overview training to be when using eVA back in your
office?
• Extremely useful
• Very useful
• Moderately useful
• Slightly useful
• Not at all useful
Below is a link to a printable job aid. Please copy the link, or right link on the and open it
in a new tab. Over the next week use this job aid as needed. You will receive a follow-up
survey in approximately one week about this job aid.
Link to printable job aid:
https://drive.google.com/open?id=1tPkXKWH8RXDh0o_RhiPVBrSFEorFaUod
OR
Below is a link to a digital job aid. Please copy the link, or right link on the and open it in
a new tab. Over the next week use this job aid as needed. You will receive a follow-up
survey in approximately one week about this job aid.
Link to digital job aid:
https://drive.google.com/open?id=1ZPQu9iAMdgTKhXDVfD2RyZX8jH1qIZTVoOOBjIE1vb8
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 27
Appendix B
Survey Two
Approximately how long ago did you complete the eVA Overview Course?
• One Month
• Two Months
• Three Months
• Four Months
• Five Months
• Six Months
What is your age?
• 18 - 29
• 30 - 39
• 40 - 49
• 50 - 59
• 60 - 69
• 70+
Which job aid did you use?
• The printable job aid
• The digital job aid
How frequently did you use the job aid?
• Never
• Once
• Twice
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 28
• Three times
• Four Times
• Five or more times
Did the job aid assist you with using eVA in your workplace?
• A great deal
• A lot
• A moderate amount
• A little
• None at all
How did the job aid assist you with the transfer of the eVA Overview training into using
eVA in your workplace?
• Rank these components of training from most preferred to least preferred for the type of
training you get at work.
• Being away from my desk and distractions while learning
• Real-time feedback
• Having an engaged instructor
• Not being overloaded with too much information at once
• An authentic learning environment
• Being able to self-direct the learning experience
• Being able to see the practical relevance
• Being able to access content and information easily and when needed
• Total length of the training
• Having performance support tools, such as job aids
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 29
Appendix C
Identification of Investigators & Purpose of Study
You are being asked to participate in a research study conducted by Dean Stubbs from James
Madison University. The purpose of this study is to determine the impact of job aids on the
transfer of learning in the workplace along with preferred characteristics of training. This study
will contribute to the researcher’s completion of his master’s thesis.
Research Procedures
This study consists of two online surveys that will be administered to individual participants
through email using Qualtrics, an online survey tool). The second survey will be sent out in
approximately one week after the first one. You will be asked to provide answers to a series of
questions related to determining the impact of job aids on the transfer of learning and the preferred
characteristics of training.
Time Required
Participation in this study will require 10 - 20 minutes of your time for both surveys.
Risks
The investigator does not perceive more than minimal risks from your involvement in this study
(that is, no risks beyond the risks associated with everyday life).
Benefits
Potential benefits from participation in this study include the improved design of future trainings.
Confidentiality
The results of this research will be presented to the eVA stakeholders at JMU and in Richmond.
While individual responses are anonymously obtained and recorded online through the Qualtrics,
data is kept in the strictest confidence. No identifiable information will be collected from the
participant and no identifiable responses will be presented in the final form of this study. All data
will be stored in a secure location only accessible to the researcher. The researcher retains the right
to use and publish non-identifiable data. At the end of the study, all records will be destroyed.
Final aggregate results will be made available to participants upon request.
Participation & Withdrawal
Your participation is entirely voluntary. You are free to choose not to participate. Should you
choose to participate, you can withdraw at any time without consequences of any kind.
However, once your responses have been submitted and anonymously recorded you will not be
able to withdraw from the study.
Questions about the Study
THE IMPACT OF JOB AIDS ON THE TRANSFER OF LEARNING 30
If you have questions or concerns during the time of your participation in this study, or after its
completion or you would like to receive a copy of the final aggregate results of this study, please
contact:
Researcher’s Name: Dean Stubbs Advisor’s Name: Dr. Cheryl Beverly
Educational Technology Learning, Technology, and Leadership Education
James Madison University James Madison University
stubbsdd@jmu.edu (540) 568-6262
beverlcl@jmu.edu
Questions about Your Rights as a Research Subject
Dr. Taimi Castle
Chair, Institutional Review Board
James Madison University
(540) 568-5929
castletl@jmu.edu
Giving of Consent
I have been given the opportunity to ask questions about this study. I have read this consent and I
understand what is being requested of me as a participant in this study. I certify that I am at least
18 years of age. By clicking on the link below, and completing and submitting this anonymous
survey, I am consenting to participate in this research.
LINK TO SURVEY
_______Dean Stubbs_______ ___2/##/2020____
Name of Researcher Date
This study has been approved by the IRB, protocol # .
top related