knowledgeable independent focused · source: convolutional deep belief networks for scalable...

ATONRÂ PARTNERS SA 12, rue Pierra Fatio - 1204 GENEVA – SWITZERLAND - Tel: + 41 22 310 15 01 www.atonra.ch

ARTIFICIAL INTELLIGENCEThe Technology Of The Future

Knowledgeable

Independent

Focused

15 June 2017

Artificial Intelligence – The Technology of The Future

➢ 2004: AtonRâ Partners finds its roots in fundamental equity research

➢ 2014: AtonRâ Partners’ business shifts toward asset management

➢ Research implemented into thematic investments

➢ Focus on growth, innovation and technology

➢ Scientific research at the core of our DNA

➢ CHF 200 million in assets under management

2

Who We Are

Artificial Intelligence – The Technology of The Future 3

Mobile Payments

Now entering mass adoption phase

Biotechnology

New drug technologies for today’s

diseases

Innovation

Technologies transforming economy

Artificial Intelligence and Robotics

The third step of technological evolution

Global Defense and Security

Strong defense for enduring peace

Bionics

Making human enhancement a reality

Our Investment Themes


AtonRâ’s scientific advisors

Christoph Sinhart

➢ Education

✓ 2013 - 2015: Master of Science in Artificial Intelligence - Maastricht University

✓ 2010 - 2013: Bachelor of Science in Knowledge Engineering - Maastricht University

➢ Working Experience

✓ 2016: Cofounding Stainly (Switzerland) together with AtonRâ Partners

• Natural Language Processing algorithms for the financial sector

✓ 2015: Start of collaboration with AtonRâ Partners

• Developing of internal ERP System for AtonRâ Partners and trying to automate tasks

of Financial Advisors/Analysts

• IT and AI Consulting for AtonRâ PartnersChristopher Wittlinger


Sjoerd van Steenkiste

➢ Education

✓ PhD student, Artificial Intelligence, Swiss AI Lab IDSIA (2016 - present)

✓ MSc, Artificial Intelligence, Maastricht University (2013-2016)

✓ MSc, Operations Research, Maastricht University (2013-2015)

✓ BSc, Knowledge Engineering, Maastricht University (2010 - 2013)

➢ Working Experience

✓ AtonRâ Partners, Scientific Advisor (2014 - present)

✓ NNAISENSE, Research Scientist (April – June 2016)

➢ Publications & pre-prints

✓ A Wavelet-based Encoding for Neuroevolution - Sjoerd van Steenkiste, Jan Koutník, Kurt Driessens,

Jürgen Schmidhuber - Genetic and Evolutionary Computation Conference (GECCO), 2016

✓ Neural Expectation Maximization - Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber - International

Conference on Learning Representations (ICLR), 2017, workshop


AI, A Game-Changer For Many Industries

➢ One of the most impactful technologies of the 21st

century

➢ The IT industry has already been transformed by AI over

the last decade

➢ Large investments in AI have led to new industries and

smarter applications

➢ Current technology already enables other industries to be

transformed

➢ As research advances and applications become smarter,

AI is expected to become essential in our everyday life

Sources: Toptal, get.com, Google Translate, hiteks.com, hardbaconmedia, extremetech.com

Recommendation /

Advertising Systems

Spam /

Fraud Detection

Customer

Interaction

Healthcare

Self-driving

Cars /

Robotics

Language

Translation

http://get.com

http://hiteks.com

http://extremetech.com


What Is Artificial Intelligence?

➢ AI: Intelligence exhibited by machines

✓ A fundamental research area with potentially broad

practical impact

✓ Subfields correspond to goals that require intelligent

solutions

➢ Recent advances in AI are mainly the result of innovations in

Machine Learning (ML)

➢ Long-term goal is Artificial General Intelligence (AGI) or

“Strong”AI

✓ Progress in ML operates at a different scale than progress

in AGI

Learning

Planning

General

Intelligence

Reasoning

Perception

Motion &

Manipulation

Creativity

Natural

Language

Processing

Artificial Intelligence


The Need For Machine Learning

➢ Most of the human knowledge and skills can not be described in

an explicit form

✓ Large variability of inputs

✓ Abstract notion of concepts

➢ Algorithmic tasks are well-defined and can be implemented

efficiently

✓ Sorting lists, arithmetic operations

➢ Cognitive tasks can not be implemented efficiently

✓ How do we define a cat?

How to sort a list?

[4, 2, 1, 9, 6]

[2, 4, 1, 9, 6]

[1, 2, 4, 9, 6]

[1, 2, 4, 9, 6]

[1, 2, 4, 6, 9]

Which images contain a cat?

Source: image-net.org

http://image-net.org


Supervised Learning At The Heart of AI

➢ Using Machine Learning (ML), we are able to learn these complex non-linear functions / programs

that are not algorithmic in nature

➢ Current ML applications make use of supervised learning

➢ A lot of labeled data need to be available in order to ensure generalization

✓ Generalization – the ability of the machine to produce sensible answers on new inputs that it

has never seen during training

Input Machine Learning Model Output

CatCat

DogCat

CatCat

Target

DogCat

CatDog

DogCat

?


Deep Neural Networks Are A Powerful Method

➢ Researchers have succeeded in training deep neural networks

➢ They consist of many layers of differentiable non-linear transformations

✓ Learn a hierarchical feature representation along side the objective and can be trained

end-to-end

• Early layers learn low-level features such as edges and corners

• Later layers combine these features to obtain “face” features or “ear” features

• The last layer of the model then computes the output from this high-level

representation

Input Deep Learning Model Output

CatCat

DogCat

CatCat

Target

DogCat

CatDog

DogCat

Input LayerL1 L2 L3 Output

Source: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y. Ng, ICML 2009


The Deep Learning Era

➢ Deep Learning (DL) models that can be trained end-to-end have successfully rivalled human

performance in several domains

➢ These recent advances are the product of several critical components:

✓ Increased computational efficiency

✓ Availability of large amounts of labeled data to train

✓ Large funding (from industry) in machine learning research

Go Speech Recognition

Object Recognition

Atari Games Skin Cancer Detection


It’s All About Computing Power…

➢ Training Deep Learning (DL) models is computationally

expensive

✓ Optimization is iterative

✓ Learning complex functions requires models have large

numbers of parameters and big data

➢ The majority of the computations can be executed in parallel

✓ Graphic Processing Units (GPUs) are specialized at

massively parallelizing computation

✓ Chip performance has increased exponentially according to

Moore’s law

➢ GPUs have become essential for research and application in

deep learning

Source: NVIDIA


… And Data

➢ Successful industrial applications of Deep Learning (DL) are learned

through feedback and require large amounts of labeled data

➢ More complex models require more data in order to ensure

generalization performance

✓ ~1M labeled images for a 1000-way discriminator / locator of objects

in a picture

✓ ~10M labeled images of face recognition

✓ ~1B sentence pairs for Machine Translation

➢ Labelled data is the most important resource for deep learning methods

at production scale

➢ Access to the right data yields a competitive advantage

Source: https://developers.google.com/recaptcha/old/docs/customization

Acquiring labeled data through CAPTCHA

ImageNet classification


➢ Massive funding in Machine Learning/Deep Learning has sparked many breakthroughs since 2012

➢ Deep Learning is a young field and most problems benefit from an individual approach that requires some research

✓ Fundamental research can directly be incorporated in applications

✓ Tech companies have successfully acquired many researchers from academia and start-ups to perform

fundamental research

44

33

25

10

10

10

2

0

0 10 20 30 40 50

Google

Microsoft

Deepmind

Facebook

IBM

Amazon

Baidu

Apple

# papersAccepted papers at ICML 2017

Scientific Progress Continues Unabated

14


Some Challenges Remain

And Need To Be Tackled


➢ Current AI approaches are extremely good at learning complicated functions provided that enough data is available

✓ In domains where little labeled data is available, Deep Learning (DL) will likely fail

➢ DL models are mostly unable to transfer learned knowledge across different domains

➢ As a rule of thumb we can currently solve most of the problems that would require a second of thought by a human

✓ General understanding as we humans have is thus still far away

✓ There is neither a clear road to achieve this

How Powerful Are Current Methods?

16


➢ Deep Networks are black-box architectures, which makes it difficult to evaluate how a model has

learned the task

Source: worldlife.org

✓ If the network has learned about the presence of ice instead of the color of the bear skin, it will wrongly

predict the right image to be of a brown bear

➢ Generalization performance helps evaluate the learned knowledge, but is always limited

➢ This may be problematic in which an incorrect decision may affect human lives

✓ Medical diagnoses and self-driving cars are clear examples of this

Polar Bear Polar Bear Brown Bear Brown Bear Polar Bear

Train Test

Current Limitations of Deep Networks

17

http://worldlife.org/


➢ Machine Learning models do not learn concepts in the same way humans do

Source: Adversarial Examples for Semantic Image Segmentation, Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, Thomas Brox, ICLR

workshop poster. Explaining and Harnessing Adversarial Examples, Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy, https://arxiv.org/abs/1412.6572

Fooling an image classifier

Fooling an image segmenter

✓ Adversarial methods can be used to target

trained deep neural networks to alter their

predictions

✓ These so-called adversarial attacks may

seem harmless at first, but could have

disastrous effects

Fooling Learned Models, Or Hacking In An AI World

18


➢ New learning paradigms that rely less on labeled data can

overcome the limitations of supervised learning

✓ Reinforcement Learning

✓ Unsupervised Learning

➢ Increase robustness by separating out different parts of the

function to be learned

✓ Most modern self-driving car systems are not learned

end-to-end

➢ Inspecting how different parts of a model interact helps in

identifying what has been learned

➢ Many of these are active research areas

Source: Show, attend and tell: Neural image caption generation with visual attention, K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R.S. Zemel, Y. Bengio. arXiv preprint arXiv:1502.03044, Vol 2(3), pp. 5. CoRR. 2015.

input predictioninput

model

input predictionground truth

model

Unsupervised learning

Supervised learning

Reinforcement learning

Identifying what has been learned

Can We Overcome These Challenges?

input

rewardmodel worldaction

19


➢ An agent (model) observes an environment and interacts with it by means of actions

✓ The environment provides feedback to the agent in the form of rewards

✓ The agent learns to act in order to maximize its reward

➢ Many problems can be modelled in this context, i.e. control, trading, interaction

✓ However the credit-assignment problem makes learning difficult: The agent does not know, which (sequence of)

action(s) caused a positive reward

➢ There is a lot of ongoing research in this domain and current techniques are increasingly being applied in practice

Source: https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/

Reducing energy consumption of Google data centersReinforcement learning

Reinforcement Learning: Giving Rewards To AI Models

input

rewardmodel worldaction

20


➢ Learning structured representations of data without supervision

✓ These representations contain useful knowledge in a condensed form

✓ Makes it easier to generalize information across domains

✓ Supervised learning methods can learn from these representations using fewer labeled

examples

Source: Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber, Neural Expectation Maximization, ICLR workshop poster

Learning about structure in the world

inputprediction

input

model

Unsupervised learning

Unsupervised Learning Reduces The Need For Labeled Data

21


Main Fields of

Application For AI


➢ Learning generative models is useful for many application domains of AI: Feature learning, Planning

Source: StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks Han Zhang,

Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, Dimitris Metaxa https://github.com/luanfujun/deep-photo-styletransfer, https://github.com/SKTBrain/DiscoGAN

Text to image generation

Content Image Style Image Output Image

Photo Style Transfer

Media / eCommerce: Improved Creation & Design

➢ Also, interesting applications in creativity & design

✓ Generating images from text, shapes

✓ Image in-painting

✓ Cross-domain content transfer

23

https://github.com/luanfujun/deep-photo-styletransfer

https://github.com/SKTBrain/DiscoGAN


➢ AI offers many opportunities to modernize the health care industry

✓ Automated analysis of scans / treatments

✓ Automated patient diagnoses

✓ Precision medicine

➢ With the amount of labelled medical data that is already available, it is believed

that current technology could already solve specific tasks more efficiently

✓ Radiology / dermatology is the main focus of a handful of AI start-ups

➢ AI advances are also expected to aid in medical research (such as for a cure of

cancer) by predicting drug responses and searching for anomalies in large bulks

of data

Health Care: Improved Diagnostics And Personalized Medicine

24


➢ Artificial Intelligence / Machine Learning (ML) techniques are important for advanced trading techniques

✓ The large amount of data that is easily labeled makes this an interesting place for Deep Learning (DL) methods

✓ However the nature of the financial market complicates the use of DL methods

✓ The ratio of hidden market variables to observed quantities is high and requires more data to be modelled

efficiently

✓ Decision making incorporates a lot of sentiment,

which is not present in the data

Finance: Enhancing Decision-Making

➢ Deep learning models will not be able to model the

entire market

✓ However they can be used alongside financial

experts to improve upon decision making,

feature identification or anomaly detection

25


➢ Mobile devices are the primary consumer of many AI applications

➢ It is desirable to have models directly operate on the device

✓ User data can remain private

✓ Online learning from user experience yields a more personalized AI

➢ A rising number of AI applications increases the demand for hardware

➢ Cloud computing platforms that offer fast GPUs (Graphic Processing Units)

are an attractive alternative

➢ Major tech companies are developing specific AI hardware to optimize

inference

✓ Hosting this hardware in the cloud provides additional means of return on

investment

Source: https://research.googleblog.com/2017/04/federated-learning-collaborative.html

Federated Learning

Cloud Computing

Cloud operators and GPU vendors: The Picks And Shovels

of The AI Rush

26

https://research.googleblog.com/2017/04/federated-learning-collaborative.html


➢ Superintelligence could happen in decades or centuries, many of the top AI experts

disagree

✓ However recent advances in AI have revived the debate concerning AI safety

➢ Two common failure cases can be considered

✓ An AI is programmed to do harm

• Autonomous weapons - need not be robots

✓ An AI is programmed to do good but implements a destructive strategy in achieving its

goal

• Researchers / Practitioners could fail in aligning the goals of an AI with our

goals through misspecification (example: paperclip factory)

➢ The objective is to align the goals of an AI with ours before it becomes superintelligent

✓ Develop methods to evaluate what an AI has learned and be able to test its goals

✓ Learn how to raise an AI in order to have it adopt our goals and beliefs

Source: Portal 2, When Will AI Exceed Human Performance? Evidence from AI Experts Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans.

When can we expect Human Level Machine Intelligence (HLMI) as predicted by AI experts

Superintelligence & AI Safety

27


➢ A more serious and immediate concern than the superhuman intelligence scenario

➢ But we are not yet at a point where massive job losses are imminent

✓ However, AI is expected to replace a lot of educated jobs in the near future

➢ Which jobs are currently at risk?

✓ Jobs that are well-defined / have an exact outcome and for which a lot of labelled training data is

available

✓ For example translators, radiologists, taxi-drivers, planners / logistics managers

➢ The closer we get to Artificial General Intelligence, the more jobs are expected to be on the line

✓ Until we have reached that point, AI will also offer a lot of new job opportunities

Will AI Affect Jobs?

28


➢ AI will have a large impact on humanity in the 21st century

➢ Innovations in AI are currently driven by supervised machine learning

✓ Success factors include access to human expertise, compute resources and data

➢ Current supervised learning approaches are black-box systems that have some practical

disadvantages

✓ Difficult to evaluate learned knowledge, susceptible to adversarial attacks and too much

dependent on labeled data

✓ Large scale research in deep learning will mitigate these concerns in the near future

➢ Artificial General Intelligence is far away and it is unclear how to get there

➢ Job losses as a result of new AI technology are a more immediate concern

Summary

29


REINFORCEMENT

LEARNING

UNSUPERVISED

LEARNING

MACHINE

LEARNING

SUPERVISED

LEARNING

Rewards

Data

(raw data)

Labeled data

(data with tags,

e.g. dogs, cats…)

Q & A

Knowledgeable

Independent

Focused



THANK YOU FOR YOUR ATTENTION


Knowledgeable

Independent

Focused

Your team:

Stefano Rodella

Dina Fausto

Brice Mari

Antoine Walter

Claudia Collin

David Wigger

Danilina Polidori

knowledgeable independent focused · source: convolutional deep belief networks for scalable...

Documents