the cognitive science behind ottolearn...2 te cognitive science eind ottolearn ottolearn coprigt...

THE COGNITIVE SCIENCEBEHIND OTTOLEARN

v1.1

A PRODUCT OF NEOVATION LEARNING SOLUTIONS

1

THE COGNITIVE SCIENCE BEHIND OTTOLEARN OTTOLEARN

COPYRIGHT © 2018 NEOVATION CORPORATION

CONTENTSIntroduction 2

A Brief History of Adaptive Training 2

The Return of Adaptive Training 3

Gamification of Learning 4

Deep Encoding 6

Instructional Scaffolding 7

The Spacing Effect 8

Adaptive Training 9

The Testing Effect 11

The Interleaved Learning Effect 12

Applied Forgetting 13

Formative Feedback 14

When Learning Matters 16

Appendix 16

OttoLearn combines adaptive training with microlearning principles in order to first raise the learner’s level of knowledge,

and then ensure retention. Personalized training is a powerful and effective approach that assesses an individual’s

knowledge and then develops an optimal training plan unique to that learner. Adaptive training goes one step

further by continuously modifying the training plan to adapt to the performance of the learner.

2



INTRODUCTIONDisruptive technologies are critical to growth and success. When

disruptive technologies are introduced, there is a dramatic

shift in the way things are done. More often than not, these

transformations lead to new solutions, advanced ideas,

and increased productivity and profit. For example, cars were

disruptive to the blacksmithing industry, and self-driving cars will

completely disrupt how we buy and use cars.

Forward-thinking organizations overcome their fear of the

new or different and actively seek disruptive technologies to fuel

their growth.

A BRIEF HISTORY OF ADAPTIVE TRAININGIn the 1960s, when behavior scientists and educators first started

experimenting with computer-based training (CBT), they focused

on using computers to deliver personalized training. The fact that

a computer could personalize training for each learner, by

adapting to their areas of strength and weakness, was seen as the

major advantage of CBT.

One of the first adaptive training systems delivered math drills

to students over touch tone phones1. Students participating in

these adaptive practice exercises outperformed their peers.

In the decades since the birth of CBT, the mass adoption of

online training through AICC and SCORM standards improved

the interoperability of modules across learning platforms, but at

the cost of personalization. Every learner received essentially the

same training experience.

As a result, adaptive training technology became comparably

expensive and out of reach for all but the most affluent

organizations—even though it was much more effective than

non-personalized, standardized training modules.

1 https://en.wikipedia.org/wiki/Patrick_Suppes

https://en.wikipedia.org/wiki/Patrick_Suppes

3



THE RETURN OF ADAPTIVE TRAININGWith OttoLearn, any organization can harness the power

of adaptive training. Our goal is to improve training and

learning efficiency by making adaptive training available

to everyone—for the benefit of both the organization and

the learner.

OttoLearn leverages key theories from cognitive science to

rapidly and significantly improve initial learning and knowledge

retention, including:

› Gamification of learning

› Deep encoding

› Instructional scaffolding

› The spacing effect

› Adaptive training

› The testing effect

› The interleaving effect

› Applied forgetting

› Formative feedback

These learning theories, combined with big data analysis and machine learning techniques, deliver an extremely effective adaptive microlearning solution.

4



Drive toMastery

Engagement

Mechanics

Learning

Mechanics

Assesment

Mechanics

GAMIFICATION OF LEARNINGUnlike game-based learning, which actually has learners play a

video game, gamification of learning is the process of using game

mechanics to engage learners and leverage their intrinsic

motivation to achieve mastery on a topic.

OttoLearn leverages three types of game mechanics—engagement

mechanics, learning mechanics and assessment mechanics—which

combine to create an atmosphere where the learner wants to

learn and achieve mastery.

ENGAGEMENT MECHANICSFor learners to learn, they first must engage. Traditional learning

platforms are designed around a one-and-done usage pattern,

which can decrease learner engagement.

OttoLearn promotes high engagement by:

› Creating a regular, daily training cadence, which normalizes training sessions into the learner’s routine

› Delivering quick learning bursts of 2-3 minutes each

› Allowing the learner to do on-demand "extra practice" sessions

› Providing continuous physical stimulation through interactivity

› Exercising content at the edge of learner's knowledge

› Allowing the learner to engage with any device—mobile, tablet, or desktop

5



LEARNING MECHANICSWe define learning mechanics as the

processes focused on having learners

actually learn. When learners feel

themselves learning, they are

motivated to continue learning.

OttoLearn’s learning mechanics include:

› Engaging learners in Activities immediately

› Providing learners with the freedom to fail

› Eliminating the need for pre-learning

› Providing a scaffolded learning experience

› Delivering learning sessions over time

› Interleaving content to increase learning efficiency

ASSESSMENT MECHANICSWith every interaction, OttoLearn builds a model of each

learner’s mastery, which it then uses to adapt the delivery

of future Activities.

OttoLearn performs continuous assessment through

multiple mechanisms, including:

› Focusing on mastery and competence versus completion

› Algorithmically determining the probability that a learner is guessing at an answer

› Allowing for self-assessment through confidence selection

› Providing learners with formative feedback to guide self-reflection

› Leveraging applied forgetting to optimize Concept selection

GAMIFICATION OF LEARNING

6



DEEP ENCODINGUnlike traditional eLearning which organizes content in a linear

format, OttoLearn organizes content as a map. The main

container is a Module, which contains multiple Topics, each

containing multiple Concepts. A Concept is a particular unit

of knowledge.

For instance, within a Module on online security, Password Security

may be a Topic, and Designing a Secure Password may be a Concept

within that Topic:

What is the minimumpassword length? (Activity)

Which of the following is required for a password to be secure? (Activity)

Which password is most secure? (Activity)

Since a Concept is more robustly learned when it is exercised

from multiple perspectives and contexts, multiple Activities

are created for each Concept. This is important for generating

an accurate sense of the learner’s true mastery of the knowledge

within the Concept.

By mapping each Concept to multiple Activities, the learner

is able to create multiple associations and exercise the

Concept’s knowledge through multiple vectors, which

promotes long-term retention.

For the learner to demonstrate mastery in a Concept, such as

Designing a Secure Password, the learner must demonstrate

proficiency in multiple related Activities:

Protecting Your Password (Concept)

Using Two-factor Authentication (Concept)

Designing a SecurePassword (Concept)

Online Security(Module)

Designing a SecurePassword (Concept)

Phishing (Topic) Malware (Topic)

Password Security(Topic)

7



Scaffolding Description

Mastery

Prerequisite

The learner must first master one or

more Modules, Topics, or Concepts

prior to receiving more advanced

content.

Explored

Prerequisite

The learner must exercise at least

one Activity for a Concept, prior to

receiving an Activity from a more

advanced Concept.

Prioritized

Importance

The instructional designer can set the

relative importance for each Concept,

to ensure that the most important

Concepts are mastered first.

INSTRUCTIONAL SCAFFOLDINGHistorically, students would begin an apprenticeship with little

knowledge, and through years of guided practice by a master,

eventually reach mastery themselves. This is instructional

scaffolding—the framework of supporting and guiding

learners to increase their mastery over time.

In online training, you may wish to ensure the learner

understands basic definitions, then integrate that knowledge

through practice activities, and finally have the learner solve

complex scenarios.

OttoLearn provides several methods of instructional

scaffolding, allowing you to leverage advanced instructional

techniques, such as anchored instruction, to bring the learner

from novice to master.

8



When a Concept is ready to be exercised by the learner (the

minimum duration has elapsed), it is referred to as valuable;

a proficient completion of that Activity will now increase the

learner’s Mastery Level for that Concept.

With each learning session, OttoLearn will seek to

present the learner with as many valuable Activities

as will fit within a 2-3 minute maximum duration.

THE SPACING EFFECTThe spacing effect means that learning is most efficient when

studying is spread out over time.

There are several theories seeking to explain the spacing effect,

however, studies agree that it works2 and has a significant

impact on learning. One theory is study-phase retrieval theory,

which postulates that if learners are exposed to the same

information multiple times in a small timespan, the brain

seeks to be efficient by not storing duplicate memories.

The brain stores infrequently accessed information in short-term

memory, so by exercising the information at increasing intervals,

learners are training the brain to store the information in long-term

memory instead of short-term memory.

As the learner completes Activities, OttoLearn will automatically

determine the minimum amount of time which must elapse prior to

presenting the learner with another Activity from the same Concept.

2 https://www.worklearning.com/wp-content/uploads/2017/10/Spacing_Learning_Over_Time__March2009v1_.pdf

Time

Learned100

90

80

70

60

Test 1 Test 2 Test 3 Test 4

Rete

ntio

n (%

)

You've probably experienced short-term retention after cramming for an exam—once you finish, you quickly forget the material.

Retention is improved through spaced repetition.

https://www.worklearning.com/wp-content/uploads/2017/10/Spacing_Learning_Over_Time__March2009v1_.pdf

https://www.worklearning.com/wp-content/uploads/2017/10/Spacing_Learning_Over_Time__March2009v1_.pdf

9



ADAPTIVE TRAININGPersonalized training is a powerful and effective approach

that assesses a learner's knowledge and then develops

an optimal training plan unique to that learner. Adaptive

training goes one step further by continuously modifying the

training plan to adapt to the performance of the learner.

In OttoLearn, each learner is assigned a Mastery Goal for a

knowledge area (Module/Topic/Concept). Different groups of

learners may be assigned different Mastery Goals based

on their job requirements.

Group A Group CGroup B

OTTOLEARN'S MASTERY LEVELS

Learners' knowledge is unknown.Unknown0

Learners are aware of what they don’t know.Novice1

Learners can remember being exposed to the information, but cannot predictably recall.

Aware2

Learners can predictably recall, but require some thinking time and make occasional errors.

Competent3

Learners exhibit accurate and rapid recall and make few errors.

Proficient4

Learners exhibit immediate recall and make virtually no errors.

Expert5

Mastery Goal 1

Mastery Goal 3

Mastery Goal 5

10



Every time a learner completes an Activity, OttoLearn calculates

a proficiency score by analyzing a number of data points (signals)

including accuracy, duration, and confidence. If a learner has

completed an Activity with sufficient proficiency, and the related

Concept is in a valuable state, then the related Concept Mastery

Level will increase.

ADAPTIVE TRAINING

ProficiencyAnalysis

Activity

Calculated Mastery

Adapt Future Activities

ProficiencyAnalysis

OttoLearn’s adaptive algorithms are by themselves adaptive, using big data analysis to become increasingly accurate over time.

Past Performance

Accuracy

Duration

Comparison to Other Learners

Confidence

11



THE TESTING EFFECTAlso referred to as retrieval practice or test-enhanced learning,

the testing effect states that the more often a learner retrieves

information, the more likely it is to be stored in long-term memory.

The testing effect is most pronounced when the memory being

retrieved is at the edge of exiting from short-term memory (as

described by the spacing effect).

As a learner increases their mastery on a Concept, OttoLearn

will develop a model to indicate the learner’s Strength of

Learning (SOL). If a learner demonstrates rapid proficiency,

they will have a higher SOL than a learner that takes longer to

demonstrate proficiency.

OttoLearn will use multiple factors, including a learner’s SOL,

to determine when and how often they should be presented

with Activities.

0

0.2

0.4

0.6

0.8

Repeated studying Repeated testing

1.0

Prop

orti

on R

ecal

l

The more a learner retrieves a memory, the stronger it becomes.

12



THE INTERLEAVED LEARNING EFFECTTraditional eLearning delivers content in a “blocking” fashion: a

particular learning session is focused on a single knowledge area,

and proceeds sequentially through related material.

However, if the delivery of material is mixed instead—for instance,

knowledge area B, then area C, then area A—the learner will more

deeply encode the information in each knowledge area.

0%

10%

20%

30%

40%

50%

60%

70%

80%

30 day retention with non-interleaved

delivery

30 day retention with interleaved

delivery

When OttoLearn is selecting Activities to present to the learner, it selects the most valuable Activities across multiple Topics, thus automatically leveraging the interleaved learning effect.

Retention for content delivered through interleaved learning is much higher than single content blocks.

13



APPLIED FORGETTINGIn the late 19th century, a German experimental psychologist

named Hermann Ebbinghaus pioneered the scientific study

of memory. Ebbinghaus hypothesized that memory

retention progressively declines over time in a rapid,

exponential fashion.

In his experiments, he studied the memorization of nonsense

syllables, such as “ZOF” and “WID”, by repeatedly testing himself

after various time durations. His exponential decay curve is not

a true representation of workplace learning3 where learners’

retention will be strongly influenced by their

prior knowledge.

Instead of a simplistic decay curve, OttoLearn dynamically calculates

a unique forgetting signature for every learner for every

Concept, which is based on their Strength of Learning (SOL) for that

Concept. This signature is then used to decrement the learner’s

Mastery Level over time, simulating their true forgetting curve.

Learners who demonstrate greater proficiency on a Concept

will “forget” slower than learners who take longer to

demonstrate proficiency.

3 https://www.worklearning.com/wp-content/uploads/2017/10/How-much-do-people-forget-v12-14-2010.pdf

Elapsed Time Since Learning

Rete

ntio

n (%

)

Elapsed Time Since Learning

20

0

40

60

80

100

20

0

40

60

80

100

A learner with prior knowledge will "forget" at a slower rate.

A learner with no prior knowledge will "forget" at a faster rate.

https://www.worklearning.com/wp-content/uploads/2017/10/How-much-do-people-forget-v12-14-2010.pdf

https://www.worklearning.com/wp-content/uploads/2017/10/How-much-do-people-forget-v12-14-2010.pdf

14



FORMATIVE FEEDBACKThe timing, structure, and content

of post-assessment feedback has a

strong influence on learning and retention.

As described by Bangert-Drowns et al.4,

there are five states of the learner

receiving feedback.

Prescription Description

Initial stateOttoLearn assumes that the learner initially begins with

no prior knowledge.

Search and

retrieval strategies

OttoLearn measures the learner’s retrieval time, as well

as other factors to form a model of proficiency.

ResponseOttoLearn requires learners to self-assess their

confidence in a selection in order to receive the result.

Evaluate resultOttoLearn presents specific feedback for each possible

response scenario.

Adjustments

OttoLearn automatically adjusts the learner’s Mastery

Level for a Concept after each performance, which

automatically adapts which Activities are presented to

the learner in future sessions.

4 Bangert-Drowns et al. (1991, p. 217)

15



According to multiple studies, the

content and the delivery of feedback

are factors that help promote deep

encoding and retention.

Prescription Description

Use praise sparingly,

if at all5

OttoLearn provides objective feedback to the

learner, without “loud” correct/incorrect messages

which would focus the learner on self.

Provide elaborated

feedback to enhance

learning6

OttoLearn allows the trainer to provide custom

feedback to the learner for each answer option,

allowing for highly targeted feedback.

Present elaborated

feedback in

manageable units7

OttoLearn provides specific feedback on

a per-Activity basis, enabling the learner

to focus on the feedback prior to proceeding.

Provide immediate

feedback to learners with

little prior knowledge8

OttoLearn provides immediate feedback specific

to each Activity.

5 Kluger & DeNisi (1996), Butler (1987)6 Bangert-Drowns et al., 1991; Gilman, 1969; Mason &

Bruning, 2001; Narciss & Huth, 2004; Shute, 2006),7 Mayer & Moreno, 2002; Phye & Bender, 19898 Anderson et al., 2001; Azevedo & Bernard, 1995;

Corbett & Anderson, 1989, 2001; Dihoff et al., 2003; Phye & Andre, 1989

FORMATIVE FEEDBACK

16



Learn more about each of the cognitive science theories

with these easy-to-digest summaries.

APPENDIX

› Gamification of learning9

› Deep encoding10

› Instructional scaffolding11

› The spacing effect12

› Adaptive training13

› The testing effect14

› The interleaving effect15

› Applied forgetting16

› Formative feedback17

9 https://en.wikipedia.org/wiki/Gamification_of_learning10 https://en.wikipedia.org/wiki/Elaborative_encoding11 https://en.wikipedia.org/wiki/Instructional_scaffolding12 https://en.wikipedia.org/wiki/Spacing_effect13 https://en.wikipedia.org/wiki/Adaptive_learning14 https://en.wikipedia.org/wiki/Testing_effect15 https://www.scientificamerican.com/article/the-interleaving-effect-mixing-it-up-

boosts-learning/15 https://en.wikipedia.org/wiki/Forgetting_curve17 https://www.ets.org/Media/Research/pdf/RR-07-11.pdf

WHEN LEARNING MATTERSIt’s a great time to be involved in online training. We are slowly

recovering from the impact of standardized training modules,

and moving back to adaptive training focused on efficient,

long-term knowledge retention to the benefit of both the

organization and the learner.

OttoLearn has been designed from the ground up to leverage multiple proven learning theories to be the most effective and efficient training tool possible.

https://en.wikipedia.org/wiki/Gamification_of_learning

https://en.wikipedia.org/wiki/Elaborative_encoding

https://en.wikipedia.org/wiki/Instructional_scaffolding

https://en.wikipedia.org/wiki/Spacing_effect

https://en.wikipedia.org/wiki/Adaptive_learning

https://en.wikipedia.org/wiki/Testing_effect

https://www.scientificamerican.com/article/the-interleaving-effect-mixing-it-up-boosts-learning/

https://en.wikipedia.org/wiki/Forgetting_curve

https://www.ets.org/Media/Research/pdf/RR-07-11.pdf

https://en.wikipedia.org/wiki/Gamification_of_learning

https://en.wikipedia.org/wiki/Elaborative_encoding

https://en.wikipedia.org/wiki/Instructional_scaffolding

https://en.wikipedia.org/wiki/Spacing_effect

https://en.wikipedia.org/wiki/Adaptive_learning

https://en.wikipedia.org/wiki/Testing_effect



https://en.wikipedia.org/wiki/Forgetting_curve

https://www.ets.org/Media/Research/pdf/RR-07-11.pdf

the cognitive science behind ottolearn...2 te cognitive science eind ottolearn ottolearn coprigt...

Documents