The Algorithms we use to learn about Algorithms

Karl LieberherrAhmed Abdelmeged

3/16/2011 1Open House 2011

Why do we model Scientific Communities?

• Scientific Communities create and disseminate new knowledge to help society.

• A computational model of scientific communities supports the same efforts for computational problems:– focused collaboration and competition– checking of the rules of a scientific community– knowledge maintenance and evaluation

3/16/2011 2Open House 2011

Idea: Use Scientific Community Modelto focus scientific discourse

• Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute.

• Claims predict the outcome of a refutation protocol.

• Parameterized by two structures: Domain and Protocol.

• Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r.

3/16/2011 3Open House 2011

Designers• SCG• Domain

– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like

• standard solvers and solvers for niches.• providing instances with “interesting” solutions

• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.

• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)

• Avatar– researchers, developers– students

3/16/2011 4Open House 2011

Confidence

• Proposer of a claim attaches a confidence in [0,1] to the claim.

• The scholar's confidence reflects the amount of effort made by the scholar to refute the claim. If it is a mathematical claim, it is the amount of effort spent to try to prove the claim (i.e. turning it into a theorem)

3/16/2011 Open House 2011 5

Refutation

• All scholars start with same reputation. Reputation is zero sum.

• Alice proposes, Bob opposes.• When scholar Bob successfully refutes a claim

of Alice, Bob wins reputation:– Bob + ClaimConfidence

• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence

3/16/2011 Open House 2011 6

Strengthening

• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.

• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|

• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence

3/16/2011 Open House 2011 7

Agreement

• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob

loses ClaimConfidence)– (2) Bob must refute C' = C minimally strengthened

along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses ClaimConfidence).

• Then Alice must do the same:

3/16/2011 Open House 2011 8

Agreement

– (1) Alice must defend C against Bob (if not, Alice loses ClaimConfidence)

– (2) Alice must refute C' with Bob as defender (if not, Alice loses ClaimConfidence)

• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).

3/16/2011 Open House 2011 9

Domain

• Instance (language)• Solution (language)

– boolean valid(Instance)– [0,1] quality(Instance)

• InstanceSet (language, subset of Instance)– boolean valid()– boolean belongsTo(Instance)

• Response = Instance union Solution

3/16/2011 10Open House 2011

SCG(Domain)

• Protocol (fixed language)• Claim(Domain)

– boolean strengthen(Claim other) // other strengthens this

– Domain.InstanceSet getInstanceSet()– Protocol getProtocol()– [0,1] getQuality()– [-1..1] getResult(List(Domain.Response))

3/16/2011 11Open House 2011

Refutations of Claim and !Claim are efficient

Claim: F unsatisfiable• if refuted: Bob finds

satisfying J; proof of !Claim.• If defended: baby step

towards proof of Claim.• Proof: long

!Claim: F satisfiable• if refuted: Alice does not

find satisfying J; baby step towards proof of Claim.

• If defended: proof of !Claim.

• Proof: short

3/16/2011 12Open House 2011

Roles:Alice claimsBob attempts to refute

Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t

• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.

• If defended: baby step towards proof of Claim.

• Proof short.

!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t

• if refuted: Alice does not find J; baby step towards proof of Claim.

• If defended: proof of !Claim if Bob is perfect.

• Proof short.

3/16/2011 13Open House 2011

Claim involving Experiment

Claim ExperimentalTechnique(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.

14Bionetics 2010

Our vision

• Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place.

• Participating teams get baby avatars generated for X that participate in daily competitions.

• Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner.

3/16/2011 Open House 2011 15

Conclusions

• Computational Modeling of Scientific Communities is a good idea:– foster Innovation– improve education

• STEM domains: use the web effectively• Current use:

– Algorithms class– Software development class

3/16/2011 Open House 2011 16

Thank you!

3/16/2011 Open House 2011 17

Strengthening

• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.

• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|

• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence

3/16/2011 Open House 2011 18

Agreement

• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob

loses)– (2) Bob must refute C' = C minimally strengthened

along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses).

• Then Alice must do the same:

3/16/2011 Open House 2011 19

Agreement

– (1) Alice must defend C against Bob (if not, Alice loses)

– (2) Alice must refute C' with Bob as defender (if not, Alice loses)

• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).

3/16/2011 Open House 2011 20

Refutations of Claim and !Claim are efficient

Claim: F unsatisfiable• if refuted: Bob finds

satisfying J; proof of !Claim.• If defended: baby step

towards proof of Claim.• Proof: long

!Claim: F satisfiable• if refuted: Alice does not

find satisfying J; baby step towards proof of Claim.

• If defended: proof of !Claim.

• Proof: short• Alice should never have

made the claim!

3/16/2011 21Open House 2011

Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t

• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.

• If defended: baby step towards proof of Claim.

• Proof short.

!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t

• if refuted: Alice does not find J; baby step towards proof of Claim.

• If defended: proof of !Claim if Bob is perfect.

• Proof short.• Alice should never have

made the claim!?

3/16/2011 22Open House 2011

Designers

• SCG• Domain

– includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver

• Protocol• Playground: Goal: make playground designers

configurators. • Avatar designers

3/16/2011 23Open House 2011

Example Playground DesignHighest Safe Rung

• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 24Open House 2011

Designers: what they produce

• SCG /scg– scg.cd, scg.beh– /protocol

• Java classes: Singleton Pattern

• Domain /domain– /hsr: hsr.cd, hsr.beh

• /avatar (components for avatar)

• Playground– config file: location of configuration file is given as

argument to admin

3/16/2011 25Open House 2011

Config

• Config = // to configure admin<scgCfg> SCGConfig <domainConfigWrapper> Wrap(DomainConfigI).

• Example entries:– domain CSP– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 26Open House 2011

Where can we find configuration settings

• If there is a configuration file location given to the admin– in the configuration file

• If not: the default value given in the code.

3/16/2011 27Open House 2011

3/16/2011 28Open House 2011

Designers• SCG• Domain

– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like

• standard solvers and solvers for niches.• providing instances with “interesting” solutions

• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.

• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)

• Avatar– researchers, developers– students

3/16/2011 29Open House 2011

Example Playground DesignHighest Safe Rung

• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …

3/16/2011 30Open House 2011