unpredictable coursera lecture 1 slides

8/12/2019 Unpredictable Coursera Lecture 1 Slides

1/47

Lecture 1

Basics of randomness


2/47

Plan of the Lecture

The various nuances of unpredictable

1 Defining words

2-3 History of randomness

4 The fair coin

5-6 Definition of randomness


3/47

THREE W

An exercise in analogy


4/47

Randomness

Lack of predictability

Example 1: coin tossing

Ignorance in each toss

lack of pattern in the sequence (norm

Example 2: polling

We ignore which item will be picked

Single events vs. statistics


5/47

Chance

Usually it refers to coincidence that

concurrent uncorrelated causes T. Wilder, The Bridge of San Luis Rey

Desire to see a deeper cause benea

coincidence, especially when tragic Even if both causes are not random

their conjunction may be unpredicta


6/47

Free will

Usually it refers to our choices, espe

in a moral context.

Example: nobody is looking at me,

I steal this object?

The decision is not random for me

but is unpredictable for someone w

not know me.


7/47

Common element

I dont know exactly how the coin is go

be tossed

I cant imagine all that happens aroundand that may affect me

I cant predict the behavior of the other

Lack of information ( of contr


8/47

This course

We deal with

Mathematical definitionof randomness

Randomness as a

resource (a.k.a. when

unpredictable is good) Sources of randomness

in the physical world

Science on free will

We dont deal with

Chance in evolut

Statistics, decisio

Prediction in fina

politics, sports

Destiny, fate e

Randomness an

in religions


9/47

Suggested Readings

On probabilities and statistics:

Deborah J. Bennett, Randomness (Harvard University Press, Cambridge Leonard Mlodinow, The drunkards walk (Pantheon, New York, 2008)

On chance: almost any book or newspaper article deals with it.

Thornton Wilder, The bridge of San Luis Rey, 1927:

http://en.wikipedia.org/wiki/The_Bridge_of_San_Luis_Rey
http://en.wikipedia.org/wiki/The_Bridge_of_San_Luis_Reyhttp://en.wikipedia.org/wiki/The_Bridge_of_San_Luis_Reyhttp://en.wikipedia.org/wiki/The_Bridge_of_San_Luis_Rey


10/47

HISTORY OF RANDOMN

Fast forward to the 19th century


11/47

Since Antiquity

Games Ignorance fairness

Secrecy Ignorance by the adversar

Divination but someone must know(and thus be in control)


12/47

Cardano, Liber de ludo aleae (156

Fermat-Pascal letters (1654)

Huygens, De ratiociniis in ludo ale

Probability theory (XVIII century onw

Al Kindi frequency analysis (9th ceAlberti polyalphabetic cyphers (14

(inventing and cracking codes)

Kerckhoffs principle (1883)

Mathematics begin

Games

Secrecy


13/47

The bell curveTossing many coins

(De Moivre, 1733)

Distribution of

measu

(Laplac

Certainty

We cant k

result, but

rigorously be from it

Very frequent but not universal (assumptions go into it)


14/47

Laplace, Thorie analytique des probabilit

We may regard the present state of the universe as the

past and the cause of its future.

An intellect which at a certain moment would know all for

nature in motion, and all positions of all items of which nature is co

if this intellect were also vast enough to submit the

analysis, it would embrace in a single formula the movements of

bodies of the universe and those of the tiniest atom;

for such an intellect nothing would be uncertain and

just like the past would be present before its eyes.


15/47

Of all objects, the planets are those w

under the least varied aspect. We see

determine their forms, their distances

their motions, but we can never know

their chemical or mineralogical structu

August Comte, The Positive Philosop

(1842)

Physics: all is

predictable

Statistics: basics

solidly

established

End of 19th century

The more important fundamental laws

physical science have all been discov

are so firmly established that the poss

ever being supplanted in consequenc

discoveries is exceedingly remote.

Albert Michelson, Light waves and the


16/47

Wikipedia pages:

http://en.wikipedia.org/wiki/History_of_randomness http://en.wikipedia.org/wiki/Probability

http://en.wikipedia.org/wiki/Normal_distribution

http://en.wikipedia.org/wiki/Jia_Xian

Artur Ekert, Complex and unpredictable Cardano,

http://arxiv.org/abs/0806.0485

Simon Singh, The Code Book, 1999:

http://en.wikipedia.org/wiki/The_Code_Book

Suggested Readings
http://en.wikipedia.org/wiki/History_of_randomnesshttp://en.wikipedia.org/wiki/Probabilityhttp://en.wikipedia.org/wiki/Normal_distributionhttp://en.wikipedia.org/wiki/Jia_Xianhttp://arxiv.org/abs/0806.0485http://en.wikipedia.org/wiki/The_Code_Bookhttp://en.wikipedia.org/wiki/The_Code_Bookhttp://en.wikipedia.org/wiki/The_Code_Bookhttp://arxiv.org/abs/0806.0485http://en.wikipedia.org/wiki/Jia_Xianhttp://en.wikipedia.org/wiki/Jia_Xianhttp://en.wikipedia.org/wiki/Jia_Xianhttp://en.wikipedia.org/wiki/Normal_distributionhttp://en.wikipedia.org/wiki/Normal_distributionhttp://en.wikipedia.org/wiki/Probabilityhttp://en.wikipedia.org/wiki/Probabilityhttp://en.wikipedia.org/wiki/History_of_randomnesshttp://en.wikipedia.org/wiki/History_of_randomness


17/47

HISTORY OF RANDOMN

6 messages from the 20th century


18/47

Quantum physics

Precursors 1900; fully born 1926

Intrinsic randomness in elementary

physics

one cannot specify both position and

momentum with arbitrary precision

NOT a default of measurement

Lectures 6-7


19/47

Deterministic chaos

Laplace: vast enough to submit th

data to analysis use computer

1970s: butterfly effect

Discovered studying models of the w

Even simple deterministic dynamics

become unpredictable in the long ru

Lectures 4 & 5


20/47

Chance in evolution

As far as science can say, an obs

in the past would have had hard tipredicting the rise of humans:

Macroscopic chance events: asteroi

climate changes Microscopic chance events: mutatio

Not dealt with in this course.


21/47

Statistics >> predictions

Statistics are everywhere

Polls, insurance, sports

Still, we are not in full control

Natural: earthquakes

Human: finance, terrorism

Not dealt with in this course


22/47

Predict us?

Desire to predict human behavior

19th century: recognize criminals fro

physical traits, calligraphy failu

Maybe undesirable (privacy, control

Libets experiments (1980s) Lecture 8


23/47

Information theory

Turing 1940s, Shannon 1948

Unpredictability is not always bad

randomness as a resource

Computing: efficient randomized

algorithms.

Privacy, secrecy: e.g. online transac

Lectures 2 & 3

S


24/47

Summary

1. Quantum: intrinsic ignorance in physics

2. Even with computers and without quantumignorance in the long term

3. Chance in evolution: we ignore how we g

4. The limits of statistics

5. While unpredictability creeps in nature, ou

decisions seem to become more predicta

6. Randomness as a resource (computing, p


25/47

Here are some classic popular books on the topics we left out, wh

help you to start in thinking

On chance in biological evolution:

Stephen J. Gould, Wonderful life, 1989:

http://en.wikipedia.org/wiki/Wonderful_Life_(book)

On use and misuse of statistics in finance and in life:

Nate Silver, The signal and the noise, 2012:

http://en.wikipedia.org/wiki/The_Signal_and_the_Noise

Nassim Nicholas Taleb, The Black Swan, 2007:

http://en.wikipedia.org/wiki/The_Black_Swan_(2007_book)

Suggested Readings
http://en.wikipedia.org/wiki/Wonderful_Life_(book)http://en.wikipedia.org/wiki/The_Signal_and_the_Noisehttp://en.wikipedia.org/wiki/The_Black_Swan_(2007_book)http://en.wikipedia.org/wiki/The_Black_Swan_(2007_book)http://en.wikipedia.org/wiki/The_Signal_and_the_Noisehttp://en.wikipedia.org/wiki/Wonderful_Life_(book)


26/47

THE FAI

D fi iti


27/47

Definition

Fair coin =

Alphabet: two values {0,1} = bi

Single run (toss): fully unpredic Several runs: uncorrelated

Remark: Ideal as a resource, not exemplary or universa

Ch l h b t (1 di 2 6


28/47

Change alphabet (1 die = 2.6 c

Fair die =

Alphabet: six values {1,2,3,4,5

Single run (cast): fully unpredic Several runs: uncorrelated

Alphabet of size A: multiply by log2A to get

Bits:

St ti ti f f b


29/47

Statistics of sequences of b

Long sequences have almost certainly n/2 0 and

1

1

0

1

0

0

1

0

Each sequence of length n is drawn with probabil

one sequence consisting of n 0s

n-choose-k sequences with k 1s

n sequences with only one 1


30/47

For those who are interested in deriving the approximation

, you can check out Stirling approximation , which is used to e

(Note that one has to go to the third term of the series for a no

result)

nn

nn

22

2

2))!2((

!

2 n

n

n

n

Statistics within a sequenc
http://en.wikipedia.org/wiki/Stirling's_approximationhttp://en.wikipedia.org/wiki/Stirling's_approximationhttp://en.wikipedia.org/wiki/Stirling's_approximationhttp://en.wikipedia.org/wiki/Stirling's_approximation


31/47

Statistics within a sequenc

How are d-bits strings distributed within a sequence of n

Example: n = 4096

Rule of thumb: all the strings with length d satisfying d2d


32/47

ProofLemma

Proof

The randomness dilemma


33/47

The randomness dilemma

0000000000000000

does not look random

1001001111101

looks random

but for a fair coin, they are equally probable to be

How to define randomness From the probabilities (mathematical descript

Capturing what we believe random to mean


34/47

Check out this cartoon of Dilbert:

http://dilbert.com/strips/comic/2001-10-25/
http://dilbert.com/strips/comic/2001-10-25/http://dilbert.com/strips/comic/2001-10-25/http://dilbert.com/strips/comic/2001-10-25/http://dilbert.com/strips/comic/2001-10-25/http://dilbert.com/strips/comic/2001-10-25/http://dilbert.com/strips/comic/2001-10-25/


35/47

DEFINE RANDOMN

Randomness of a sequence

Kolmogorov complexity (KC


36/47

Kolmogorov complexity (KC

a.k.a. algorithmic complex

Complexity of a sequence = the length of the sho

algorithm that can generate it.

0000000000000000 Print a list of sixteen 0

1001001111101100 Print twice a 1 followetwo 0, then five 1, one

1, two 0

Captures: randomness = lack of pattern

Martin Lf definition (1966


37/47

Martin-Lf definition (1966

A sequence is random if [equivalent statemen It passes all possible statistical tests (with fa

as ideal case)

It cannot be produced by a program shorter t

(incompressible) Inspired by Kolmogorov complexity

0000000, or the digits of , are not random

sense

Algorithmic randomness

Problem: KC is uncomputab


38/47

Problem: KC is uncomputab

Not just difficult to compute: there is no consiste

of defining the shortest algorithm.

Similar to Berrys paradox:

Hypothesis: positive integers can be ordered b

smallest number of English words needed to d

each of them. Then the smallest positive integer not definabl

less than N English words can then be describ

13 words paradox for N>13

Sketch of formal proof


39/47

Sketch of formal proof

Take k0 as input

For L=1,2,

For every s of length L

Compute K(s)

If K(s)>k0Output s L

ength

U

k0

s with K(s)>k0

Length log(k0)

Wrong for k0 large enough

S t d R di


40/47

Wikipedia pages:

http://en.wikipedia.org/wiki/Kolmogorov_complexity

http://en.wikipedia.org/wiki/Martin-Lof

http://en.wikipedia.org/wiki/Statistical_randomness

http://en.wikipedia.org/wiki/Algorithmic_randomness

The NIST 800-22 battery of statistical tests for random number ge

http://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22r

Suggested Readings
http://en.wikipedia.org/wiki/Kolmogorov_complexityhttp://en.wikipedia.org/wiki/Martin-Lofhttp://en.wikipedia.org/wiki/Statistical_randomnesshttp://en.wikipedia.org/wiki/Algorithmic_randomnesshttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdfhttp://en.wikipedia.org/wiki/Algorithmic_randomnesshttp://en.wikipedia.org/wiki/Algorithmic_randomnesshttp://en.wikipedia.org/wiki/Algorithmic_randomnesshttp://en.wikipedia.org/wiki/Statistical_randomnesshttp://en.wikipedia.org/wiki/Statistical_randomnesshttp://en.wikipedia.org/wiki/Martin-Lofhttp://en.wikipedia.org/wiki/Martin-Lofhttp://en.wikipedia.org/wiki/Martin-Lofhttp://en.wikipedia.org/wiki/Martin-Lofhttp://en.wikipedia.org/wiki/Kolmogorov_complexityhttp://en.wikipedia.org/wiki/Kolmogorov_complexityhttp://en.wikipedia.org/wiki/Kolmogorov_complexity


41/47

DEFINE RANDOMN

Randomness of the process

Unpredictable process


42/47

Unpredictable process

1001001111101100

Can anyone predict the

The next million bits may all be 0

but of course, if I have never seen a 1, I wont trus

process to be random

Not-too-black boxes


43/47

algorithm

Not too black boxes

?1001001111101100

The full black-box model may be too strong:

Kerckhoff-Shannon: no security by obscurity, let the algo

known, but not the seed.

Example (not very practical)

Algorithm: the sequence is drawn from the second letter of ea

book Pretty unpredictable if one cant guess which book.

seed

Pseudorandomness


44/47

Pseudorandomness

?1001001111101100

More practical: mechanical algorithms:

Mathematical function (seed: e.g. some divider)

Scramble the last digits of the clock of the computer (seed:instant the key is touched)

algorithm

seed

This is pseudo-random: nothing is reallyrandom, but it is unpre

someone who does not know the seed. Is it really?

Pseudorandomness (2)


45/47

Pseudorandomness (2)

algorithm ?1001001111101100

Can the seed be inferred by observing the process?

YES it can but if things are well done, only with high compu

power

seed

Very frequent definition: the process is random if it is unpre

with limited computational power[more precise Lectures

At a glance


46/47

At a glance

Random sequence

No pattern

May not be secret: it

could have been

recorded once and

copied many times

Practice: uncomputable;

trust a finite battery of

tests

Random process

If randomness isgenerated, it is s

May (occasionally

a simple sequen

Practice: need to

characterization o

process

Summary of Lecture 1


47/47

Summary of Lecture 1

Randomness = lack of predictability Lack of information lack of control

Mathematical tool: statistics

Fair coin as ideal case Definition of randomness

Of a sequence? Of a process?

Verification requires some element of trust

The various nuances of unpredictable

unpredictable coursera lecture 1 slides

Documents