how can artificial intelligence help in workplace learning · how can artificial intelligence help...

TU-E3110, Work Design in Organizations

Antti Keurulainen, Lic.Sc. Chairman at BitvillePhD Candidate at Aalto University

27th March 2018

How can artificial intelligence help in workplace learning

Presentation outline• Terms

• What are machine learning and artificial intelligence

• Supervised learning, unsupervised learning and reinforcement learning

• AI and machine learning history and milestones

• Artificial neuron and artificial neural network

• Machine learning and AI for human learning

• Case: sentiment analysis

• Case: deep knowledge tracing

• Case Automated essay grading

• Future scenarios

How can artificial intelligence help in workplace learning | Antti Keurulainen | Bitville

Machine learning and Artificial intelligence

Deep neural nets and AI are trending


Source: Google trends 21st March 2018

Machine learning“The field of study that gives computers the ability to learn without being explicitly programmed” (Arthur L. Samuel 1959)

“System that learns from data”

In more descriptive terms, machine learning offers automated methods to analyze data, and involves a capability to learn from previous data or experience.


DataAlgorithm that

learns from data

Output

Supervised machine learning

data

target

prediction

Artificial intelligence (AI)Any human-made device or system that perceives its environment and makes decisions and/or takes actions that maximize its chance of success at some goal.


Example of an AI milestone (1997)Garry Kasparov was beaten in chess by IBM Deep Blue computing system.


Example of an Machine learning milestone (2016)Lee Sedol was beaten in Go by Google Deepmind “Alpha Go” system.

How can artificial intelligence help in workplace learning | Antti Keurulainen | Bitville Geordie Wood

AI and machine learning


Artificial Intelligence

Systems that learn from data

Supervised and unsupervised machine

learning

Systems that learn from experience

Reinforcement Learning

Rule-based systems

AI and machine learning


Artificial Intelligence

Systems that learn from data

Supervised and unsupervised machine

learning

Systems that learn from experience

Reinforcement Learning

Rule-based systems

Deep reinforcement learning, dynamic structure decision trees, dynamic structure expert systems, neural random forest,…


Supervised learning

Unsupervised learning

Reinforcement learning


Supervised learning

TrainingTypically, a big number of data samples and target values are given to the system

After the training the system is able to:

• Classify the unseen examples (classification, categorization)

• Estimate a value (regression)

• Reduce noise

• etc


Unsupervised learning

TrainingTypically, a big number of samples without target values are given to the system

After the training the system is able to:

• Cluster samples in groups (clustering)

• Detect an unnormal sample (anomaly detection)

• Form representations (e.g. word embeddings)

• Generate more content like samples

• etc


Reinforcement learning

Agent Environment

action

state

reward

TrainingBy certain type of trial and error procedure, the system is able to learn proper actions that help achieving the goal.

Machine intelligence has a long history



Wang & Raj (Carnegie Mellon University 2017)

On the Origin of Deep Learning

Why is now the time for machine intelligence?• A lot of data available (big data)

• A lot of computing power (cloud computing, affordable GPUs, …)

• New efficient libraries (Python packages, Tensorflow, Theano, Keras, …)

• New algorithms and advances in machine learning


Examples of successful AI fields

• Computer Vision

• Natural Language Processing (NLP)

• Robotics

• Generative systems

• Game AI

• Anomaly detection


The goals of AI and machine learning• Artificial general intelligence

• Possibility for an intelligent dialogue between humans and machines

• Machine reasoning

• Solve major problems

• etc


AI in education

AIAI in

education (AIED)

AI in education is in the intersection between the AI technology, human psychology and neuroscience

Psychology

Neuroscience


Interesting goals in workplace training• Personalized learning for all• Learning as a group• Sharing tacit knowledge in organizations• Generating new knowledge• Refining existing knowledge• Assessing learning level and rate• Building and maintaining learning motivation• Offering recommendations• Crowd creation• Speed to competence


Active research fields supporting AIED• Natural language processing (NLP)

• Memory augmented neural networks (MANN)

• Machine meta-learning

• Machine cognition

• Human-in-the-loop reinforcement learning

• Multiagent behaviour analysis

• Multiagent communication


Answer

Question

$

Kirkpatrik model for evaluating learning solutions


The first level, reaction, describes how the learners themselves thought and felt about the learning.

The second level, learning, tries to measure the actual learning or change in the attitudes

The third level, behavior, evaluates whether the behavior of the learner really changed after the training.

The highest level in the Kirkpatrick model, results, describes the actual value that the training produced for the organizer.

Three AI for workplace learning cases

Free text sentiment analysis

Kirkpatrick level 1

Knowledge tracing

Kirkpatrick level 2

Automated essay gradingKirkpatrick level 2-3

Sentiment analysis• Sentiment analysis is a collection of

methods with the main intent to observe the opinion or attitude, for example, of a sentence expressed in natural language.

• Typical solution is based on training the neural network with movie reviews, tweets, or some other available dataset. Word vectors and neural network can then be further trained for specific task with smaller dataset.


IMDB movie review database

Training a system to solve one task and use it to solve some other task.

Transfer learning

Train a system to estimate the sentiment of movie reviews

Use the system to estimate the sentiment of course feedback text

from students


Test data


ℎ = 𝜙𝜙 �𝑖𝑖=0

𝑛𝑛

𝑤𝑤𝑖𝑖𝑥𝑥𝑖𝑖

Artificial neuron

Multilayer Perceptron (MLP) network

The MLP networks are the most ”basic” form of deep neural networks.


Some resultsModel Training set Test set Accuracy Notes

Skip-gram + GRU Word2vec: aclIMDBSentiment analyzer: rt-polaritydata

Course feedbacks 73% Improved with dropout and larger training set for word2vec

Skip-thoughts + linear sentiment analyzer

Skip thoughts: rt-polaritydata

Course feedback 78%

Skip-gram + linear Word2vec:rt-polaritydataSentiment analyzer:rt-polaritydata

Course feedbacks 70,4% Window size 3, embedding dim 100.Average over 3 runs(69,8%-71,2%)

Skip-gram + GRU with continued word2vec training

aclIMDB (30000) aclIMDB (20000) 81.3% Does not generalize well to course feedbacks


Knowledge tracing

In knowledge tracing, the goal is to predict users' ability to solve the next exercise given his or her past behaviour and similar data from other users.

Recurrent networks

Because of the recurrent loops, the recurrent neural networks have a “memory-like” function. That makes them useful for any sequential data.

LSTM networks are one type of recurrent networks, that include some rather sophisticated internal functionality.


Recurrent network for knowledge tracing

Piech, Chris et al, Deep Knowledge tracing, NIPS 2015


Essay grading

In automated essay grading, the target is to predict the grading that a human grader would give to an essay

Example: end-to-end memory networks

Sukhbaatar et al et al. End-to-end memory networks, 2015How can artificial intelligence help in workplace learning | Antti Keurulainen | Bitville

Example: Memory augmented networks for essay grading

• The model was able to reach state-of-the-art results in 7 out of 8 essay sets in the Kaggle ASAP dataset

• This dataset includes around 22 000 digitized long essays (150-500 words) and 27 000 short essays (around 50 words) with human annotated grading labels

Xiaolu Xiong et al.Going Deeper with Deep Knowledge TracingProceedings of the 9th International Conference in Educational Data Mining, 2016


Our essay grading ensamble system

Fully connected neural network for combining lower level neural network results

Memory augmented neural network for style

grading

Recurrent neural network for content relevance

grading

Essay Essay

Grade


Production environment overview

Model training in FloydHub cloud using Tensorflow

Hosting neural network parameters and word vectors in a parameter server

Conducting real-time inference in browser by using deeplearn.js

1 2 3


Essay grading system demo

video

assessment

poll

essay

feedback

Week 1 Week 2 Week 3

Knowledge tracing module collects interaction data and assess activity/competence levels

AI scheduler finds the best times for content delivery

AI essay grader assess the essays and gives labels for other AI functions

AI sentiment analyzer

AI recommendation engine selects best content for personalized learning experience

Example: Microlearning model and AI


Continuously improving teaching model

Deliver content

Observe outcome

Learn better training model and

learner model

AI – based learning

management


Examples of future directions

I Mordatch S Milli, P Abbeel. Interpretable and pedagogical examples. arXivpreprint arXiv:1711.00694, 2017.

Pedagogical concept learned by machine learning

Picture source: https://blog.openai.com/interpretable-machine-learning-through-teaching/

• Human teacher give intuitively most informative examples of the concepts to students

• ”Teacher” and “student” are deep neural networks, trained sequentially instead of jointly

• ”Teacher” learns to give interpretable pedagogically good examples for various concepts.

• The pedagogical concept is tested to humans by using the Mechanical Turk service


Foerster, Jakob N, Chen, Richard Y, Al-Shedivat, Maruan, Whiteson, Shimon, Abbeel, Pieter, and Mordatch, Igor. Learning with opponent-learning awareness. arXiv preprint arXiv:1709.04326, 2017.

Machine Theory of Mind

System architecture and example pictures from the research paper

• ToMnet learns to model the beliefs and goals of an deep reinforcement learning agent.

• Can model the agents’ characteristics and mental states using only a small number of observations.

• Can recognize false beliefs of the agent

• Uses a metalearning concept


how can artificial intelligence help in workplace learning · how can artificial intelligence help...

Documents