The Algorithms we use to learn about Algorithms
Karl LieberherrAhmed Abdelmeged
3/16/2011 1Open House 2011
Why do we model Scientific Communities?
• Scientific Communities create and disseminate new knowledge to help society.
• A computational model of scientific communities supports the same efforts for computational problems:– focused collaboration and competition– checking of the rules of a scientific community– knowledge maintenance and evaluation
3/16/2011 2Open House 2011
Idea: Use Scientific Community Modelto focus scientific discourse
• Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute.
• Claims predict the outcome of a refutation protocol.
• Parameterized by two structures: Domain and Protocol.
• Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r.
3/16/2011 3Open House 2011
Designers• SCG• Domain
– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like
• standard solvers and solvers for niches.• providing instances with “interesting” solutions
• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.
• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)
• Avatar– researchers, developers– students
3/16/2011 4Open House 2011
Confidence
• Proposer of a claim attaches a confidence in [0,1] to the claim.
• The scholar's confidence reflects the amount of effort made by the scholar to refute the claim. If it is a mathematical claim, it is the amount of effort spent to try to prove the claim (i.e. turning it into a theorem)
3/16/2011 Open House 2011 5
Refutation
• All scholars start with same reputation. Reputation is zero sum.
• Alice proposes, Bob opposes.• When scholar Bob successfully refutes a claim
of Alice, Bob wins reputation:– Bob + ClaimConfidence
• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence
3/16/2011 Open House 2011 6
Strengthening
• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.
• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|
• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence
3/16/2011 Open House 2011 7
Agreement
• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob
loses ClaimConfidence)– (2) Bob must refute C' = C minimally strengthened
along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses ClaimConfidence).
• Then Alice must do the same:
3/16/2011 Open House 2011 8
Agreement
– (1) Alice must defend C against Bob (if not, Alice loses ClaimConfidence)
– (2) Alice must refute C' with Bob as defender (if not, Alice loses ClaimConfidence)
• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).
3/16/2011 Open House 2011 9
Domain
• Instance (language)• Solution (language)
– boolean valid(Instance)– [0,1] quality(Instance)
• InstanceSet (language, subset of Instance)– boolean valid()– boolean belongsTo(Instance)
• Response = Instance union Solution
3/16/2011 10Open House 2011
SCG(Domain)
• Protocol (fixed language)• Claim(Domain)
– boolean strengthen(Claim other) // other strengthens this
– Domain.InstanceSet getInstanceSet()– Protocol getProtocol()– [0,1] getQuality()– [-1..1] getResult(List(Domain.Response))
3/16/2011 11Open House 2011
Refutations of Claim and !Claim are efficient
Claim: F unsatisfiable• if refuted: Bob finds
satisfying J; proof of !Claim.• If defended: baby step
towards proof of Claim.• Proof: long
!Claim: F satisfiable• if refuted: Alice does not
find satisfying J; baby step towards proof of Claim.
• If defended: proof of !Claim.
• Proof: short
3/16/2011 12Open House 2011
Roles:Alice claimsBob attempts to refute
Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t
• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.
• If defended: baby step towards proof of Claim.
• Proof short.
!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t
• if refuted: Alice does not find J; baby step towards proof of Claim.
• If defended: proof of !Claim if Bob is perfect.
• Proof short.
3/16/2011 13Open House 2011
Claim involving Experiment
Claim ExperimentalTechnique(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.
14Bionetics 2010
Our vision
• Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place.
• Participating teams get baby avatars generated for X that participate in daily competitions.
• Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner.
3/16/2011 Open House 2011 15
Conclusions
• Computational Modeling of Scientific Communities is a good idea:– foster Innovation– improve education
• STEM domains: use the web effectively• Current use:
– Algorithms class– Software development class
3/16/2011 Open House 2011 16
Thank you!
3/16/2011 Open House 2011 17
Strengthening
• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.
• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|
• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence
3/16/2011 Open House 2011 18
Agreement
• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob
loses)– (2) Bob must refute C' = C minimally strengthened
along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses).
• Then Alice must do the same:
3/16/2011 Open House 2011 19
Agreement
– (1) Alice must defend C against Bob (if not, Alice loses)
– (2) Alice must refute C' with Bob as defender (if not, Alice loses)
• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).
3/16/2011 Open House 2011 20
Refutations of Claim and !Claim are efficient
Claim: F unsatisfiable• if refuted: Bob finds
satisfying J; proof of !Claim.• If defended: baby step
towards proof of Claim.• Proof: long
!Claim: F satisfiable• if refuted: Alice does not
find satisfying J; baby step towards proof of Claim.
• If defended: proof of !Claim.
• Proof: short• Alice should never have
made the claim!
3/16/2011 21Open House 2011
Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t
• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.
• If defended: baby step towards proof of Claim.
• Proof short.
!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t
• if refuted: Alice does not find J; baby step towards proof of Claim.
• If defended: proof of !Claim if Bob is perfect.
• Proof short.• Alice should never have
made the claim!?
3/16/2011 22Open House 2011
Designers
• SCG• Domain
– includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver
• Protocol• Playground: Goal: make playground designers
configurators. • Avatar designers
3/16/2011 23Open House 2011
Example Playground DesignHighest Safe Rung
• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 24Open House 2011
Designers: what they produce
• SCG /scg– scg.cd, scg.beh– /protocol
• Java classes: Singleton Pattern
• Domain /domain– /hsr: hsr.cd, hsr.beh
• /avatar (components for avatar)
• Playground– config file: location of configuration file is given as
argument to admin
3/16/2011 25Open House 2011
Config
• Config = // to configure admin<scgCfg> SCGConfig <domainConfigWrapper> Wrap(DomainConfigI).
• Example entries:– domain CSP– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 26Open House 2011
Where can we find configuration settings
• If there is a configuration file location given to the admin– in the configuration file
• If not: the default value given in the code.
3/16/2011 27Open House 2011
3/16/2011 28Open House 2011
Designers• SCG• Domain
– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like
• standard solvers and solvers for niches.• providing instances with “interesting” solutions
• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.
• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)
• Avatar– researchers, developers– students
3/16/2011 29Open House 2011
Example Playground DesignHighest Safe Rung
• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 30Open House 2011