info introduction to...

[email protected]/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

introduction to informaticslecture 3


INDIANAUNIVERSITY

I501introduction

to informatics


Readings until now

This week Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters: 1 and 2 Gleick, J. [2011]. The Information: A History, a Theory, a Flood. Random

House. Chapter 8. Optional

McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133.

Heims, S.G. [1991]. The Cybernetics Group. MIT Press. Chapters 11, and 12. Presentations

Freeth, Tony. 2014. “Eclipse Prediction on the Ancient Greek Astronomical Calculating Machine Known as the AntikytheraMechanism.” PloS One 9 (7): e103275. Plank, Nicholas

Wasserman, M., X.H.T. Zeng, and L.A.N. Amaral [2015]. “Cross-evaluation of metrics to estimate the significance of creative works”. PNAS 112 (5) 1281-1286. Boothby, Clara

Lecture Notes The Nature of Information

Available http://canvas.iu.edu and listed at

http://informatics.indiana.edu/rocha/academics/i501 Also check out

Links and notes at http://sciber.blogspot.com/

introduction to informatics


INDIANAUNIVERSITY

I501introduction

to informatics

Informatics luis rocha 2017 The external tape as a general principle (system) of universal computing

Charles Babbage (1791 – 1871) and Ada Lovelace (1815-1852)

Analytical engine Separated memory store from a

central processing unit (or ‘mill’) Cogs not just numbers

variables Programmable

instructions on punched cards Inspired by the Jacquard Loom

Ada Lovelace: the science of operations Set of (recursive) rules for producing

Bernoulli numbers (a program) Separation of variable and operational

(data) cards would punch out cards for later use

“the Engine eating its own tail.” (Babbage)

distinction between numbers that mean thingsand numbers that do things.


INDIANAUNIVERSITY

I501introduction

to informatics


Alan Turing (1912-1954)

“The chemical basis of morphogenesis” Turing, A. M. Phil. Trans. R. Soc. Lond.

B 237, 37–72 (1952). Reaction-diffusion systems

“Computing machinery and intelligence” Turing, A. M. Mind 49, 433–460 (1950).

The “Turing Test” “On computable numbers with an

application to the Entscheidungsproblem” Turing, A. M. Proc. Lond. Math. Soc. s2–42, 230–265 (1936–37). Turing machine, universal

computation, decision problem

key contributions (most relevant to biocomplexity)

Brenner, Sydney. [2012]. “Life’s code script.” Nature 482 (7386): 461-461.


INDIANAUNIVERSITY

I501introduction

to informatics


computation as number crunching

Computing with numbers It is always possible to encode

any symbol string into a sequence of integers

A sequence of integers can be mapped into a single natural number Gödelization (prime factorization)

Computer scientists can concentrate on functions that take a single number as input and output Computation as a mapping of

numbers to other numbers Models of Computation

How to construct number mappings Turing machine, general

recursive functions, λ-calculus


INDIANAUNIVERSITY

I501introduction

to informatics


Turing’s tape

“On computable numbers with an application to the Entscheidungsproblem” Turing, A. M. Proc. Lond. Math. Soc. s2–42, 230–265 (1936–37).

Turing machine, universal computation, decision problem Machine’s state is controlled by a program, while data

for program is on limitless external tape every machine can be described as a number that can be

stored on the tape for another machine Including a Universal machine

distinction between numbers that mean things (data) and numbers that do things (program)

A fundamental principle of computation

“The fundamental, indivisible unit of information is the bit. The fundamental, indivisible unit of digital computation is the transformation of a bit between its two possible forms of existence: as [memory] or as [code]. George Dyson, 2012.


INDIANAUNIVERSITY

I501introduction

to informatics


At every discrete time instance the machine is in a single state

Program is a state transition table

state Read symbol

Next state

Write symbol

Tape move

0 4 1 - left

1 - 0 1 right


INDIANAUNIVERSITY

I501introduction

to informatics


computation

Process of rewriting strings in a formal system according to a program of rules Operations and states are syntactic Symbols follow syntactical rules Rate of computation is irrelevant

Program determines result, not speed of machine Physical implementation is irrelevant for result

Computer Physical device that can reliably

execute/approximate a formal computation Errors always exist Design aims to make rate and dynamics irrelevant

some facts

“[…] essential elements in the machine are of a binary […] nature. Those whose state is determined by their history and are time-stable are memory elements. Elements of which the state is determined essentially by the existing amplitude of a voltage or signal are called ‘gates’”. Bigelow et al, 1947


INDIANAUNIVERSITY

I501introduction

to informatics


John Von Neumann (1903-1957)

realizing the power of Turing’s tape physical (electronic) computers emphasized the importance of the stored-

program concept (the external tape) EDVAC

allows machine to modify its own program von Neumann architecture: The functional

separation of storage from the processing unit. programs can exist as data (two roles)

Converts tape to fixed-address memory (random-access memory)

Turing machines beyond the decision problem

“Let the whole outside world consist of a long paper tape”. —John von Neumann, 1948

“ ‘Words’ coding the orders are handled in the memory just like numbers” --- distinction between numbers that mean things and numbers that do things.


INDIANAUNIVERSITY

I501introduction

to informatics


John Von Neumann (1903-1957)

realizing the power of Turing’s tape physical (electronic) computers emphasized the importance of the stored-

program concept (the external tape) EDVAC

allows machine to modify its own program von Neumann architecture: The functional

separation of storage from the processing unit. programs can exist as data (two roles)

Converts tape to fixed-address memory (random-access memory)

Turing machines beyond the decision problem

“Let the whole outside world consist of a long paper tape”. —John von Neumann, 1948

“ ‘Words’ coding the orders are handled in the memory just like numbers” --- distinction between numbers that mean things and numbers that do things.

“Since Babbage’s machine was not electrical, and since all digital computers are in a sense equivalent, we see that this use of electricity cannot be of theoretical importance.… The feature of using electricity is thus seen to be only a very superficial similarity.” (Alan Turing)


INDIANAUNIVERSITY


I501introduction

to informatics

design principles of computationBabbage/Lovelace, Turing’s tape, and roles of information

distinction between numbers that mean things and numbers that do things.


INDIANAUNIVERSITY

I501introduction

to informatics


Nature.com; ANDY POTTS; TURING FAMILY


INDIANAUNIVERSITY

I501introduction

to informatics


other pre-cybernetics developments

“The Cerebral Inhibition Meeting” New York City, May 1942

Organized by Frank Freemont-Smith of the Josiah Macy Jr. Foundation

From the human sciences Lawrence Frank, Margaret Mead and Gregory Bateson

From the sciences Warren McCulloch and Arturo Rosenblueth

Result Huge excitement about Rosenblueth’s presentation of concepts

from Norbert Wiener and Julien Bigelow Homeostasis, purposeful action (goal-direction), aiming A new paradigm of interdisciplinary research?

Goal-directed actions Controversial: explaining actions in terms of future events, violating

cause and effect Teleological mechanisms

Circular causality requiring negative feedback (postulated to be very common) Present state becomes input for action at next moment: State-

determined systems The mathematics were accessible

1942 meeting


INDIANAUNIVERSITY

I501introduction

to informatics


McCulloch & Pitts

McCulloch, W. and W. Pitts [1943], "A Logical Calculus of Ideas Immanent in Nervous Activity". Bulletin of Mathematical Biophysics 5:115-133. A Turing machine program could be implemented in a finite

network of binary neuron/switches Neurons as basic computing unit of the brain Circularity is essential for memory (closed loops to sustain memory)

Brain (mental?) function as computing Others at Macy Meeting emphasized other aspects of brain

activity Chemical concentrations and field effects (not digital)

Ralph Gerard and Fredrik Bremmer

Memory can be maintained in circular networks of binary switches


INDIANAUNIVERSITY

I501introduction

to informatics


Cybernetics was born

The Feedback Mechanisms and Circular Causal Systems in Biology and the Social Sciences March 1946 (10 meetings between 1946

and 1953) Interdisciplinary

Since a large class of ordinary phenomena exhibit circular causality, and mathematics is accessible, let’s look at them with a war-time team culture

Participants John Von Neumann, Leonard Savage,

Norbert Wiener, Arturo Rosenblueth, Walter Pitts, Margaret Mead, Heinz von Foerster, Warren McCulloch, Gregory Bateson, Claude Shannon, Ross Ashby, etc.

Key concepts Homeostasis, Circular causality

requiring negative feedback (postulated to be very common)

Present state becomes input for action at next moment: State-determined systems

The mathematics were finally accessible

post-war science: the Josiah Macy Jr. Foundation Meetings


INDIANAUNIVERSITY

I501introduction

to informatics


cybernetics

Synthetic approach Engineering-inspired Supremacy of mechanism

Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology

All can be axiomatized and computed Mculloch&Pitts’ work was major influence

“A logical calculus of the ideas immanent in nervous activity”. Bulletin of Mathematical Biophysics 5:115-133 (1943).

A Turing machine (any function) could be implemented with a networkof simple binary switches (if circularity/feedback is present)

post-war science

Macy Conferences: 1946-53

Warren S. McCulloch

Claude Shannon

Margaret Mead


INDIANAUNIVERSITY

I501introduction

to informatics


other key concepts

Norbert Wiener and Arturo Rosenblueth Goal-directed behavior and negative feedback (control) Homeostasis and circular causality

In machines and biology Automata Theory (Von Neumann) Communication and Information

The fundamental idea is the message, even though the message may not be sent by man and the fundamental element of the message is the decision” (Norbert Wiener)

Shannon’s Information and Wiener’s Communication Theory Natural semiotics (McCulloch and others later get into Peircean Semiotics)

“functional equivalence” of systems (general systems) Bio-inspired mathematics and engineering and computing/mechanism-inspired

biology and social science

at the Macy meetings


INDIANAUNIVERSITY

I501introduction

to informatics


other key concepts

Gregory Bateson and Margaret Mead Homeostasis and circular causality in society

Transvestite ceremony to diffuse aggressive action in Iatmul culture

Learning and evolution Can a computer learn to learn?

A new organizing principle for the social sciences (control and communication) As much as evolution was for Biology

Lawrence Frank The new interdisciplinary concepts needed a

new kind of language Higher generality than what is used in single

topic disciplines A call for a science of systems

Yehoshua Bar-Hillel Optimism of a new (cybernetics and

information) age “A new synthesis […] was destined to open

new vistas on everything human to help solve many of the disturbing open problems concerning man and humanity”.

at the Macy meetings


INDIANAUNIVERSITY

I501introduction

to informatics


British Cybernetics

The Ratio Club (starting in1949) British cybernetics meetings

William Ross Ashby, W. Grey Walter, Alan Turing. etc “computation or the faculty of mind which calculates, plans

and reasons” Also following Wiener’s use of “Machina ratiocinatrix” in

Cybernetics (1948), following Leibniz’ “calculus ratiocinator”

Turing as cybernetician


INDIANAUNIVERSITY

I501introduction

to informatics


Shannon’s mouse

trial and error algorithm information as reduction of uncertainty in the

presence of alternatives (combinatorics) lifelike behavior

trial and error to learn path from many alternatives adapts to new situations

how is learning achieved? Correct choices, information gained from reduced

uncertainty, must be stored in memory memory of information as a design principle

of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays

Brain as (switching) machine

controlling information to achieve life-like behavior


INDIANAUNIVERSITY

I501introduction

to informatics


next class

Week 4 Lecture

Prokopenko, Mikhail, Fabio Boschetti, and Alex J. Ryan. [2009]. "An information-theoretic primer on complexity, self-organization, and emergence. “ Complexity 15(1): 11-28

Presentations & Discussion Barabasi and Albert (1999) Emergence of Scaling in

Random Networks, Science 296 (5439). Erkol, Sirag

Hofman, Jake M., Amit Sharma, and Duncan J. Watts. "Prediction and explanation in social systems." Science 355.6324 (2017): 486-488. Kaminski, Patrick

Lecture Notes The Nature of Information

Optional Aleksander, I. [2002]. “Understanding Information Bit by

Bit”. In: It must be beautiful : great equations of modern science. G. Farmelo (Ed.), Granta, London.

Readings (available in Canvas)

info introduction to...

Documents