The Scientific Community Gamefor STEM Innovation and Education
(STEM: Science, Technology, Engineering and Mathematics)
Karl LieberherrAhmed Abdelmeged
3/16/2011 1Open House 2011
3/16/2011 Open House 2011 2
SCG =ScientificCommunityGame
Why Scientific Community Game?Why are you here?
• … motives in academic publishing: – desire for recognition and respect from the people
one regards as peers, – desire to have impact (on conclusions being
reached, on the development of the discipline, etc.), and
– desire to participate in significant knowledge-building discourse.
• e.g., Scardamalia, M., & Bereiter, C. (1994)
Open House 2011 33/16/2011
Why do we model Scientific Communities?
• Scientific Communities create and disseminate new knowledge to help society.
• A computational model of scientific communities supports the same efforts for computational problems:– focused collaboration and competition– checking of the rules of a scientific community– knowledge maintenance and evaluation
3/16/2011 4Open House 2011
How it Works
• Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute.
• Claims predict the outcome of a refutation protocol.
• Parameterized by two structures: Domain and Protocol.
• Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r.
3/16/2011 5Open House 2011
Mathematics Example:Strengthen, Agree and Refute
• Alice claims Claim C(t):ForAll [x in [0,1]] Exists[y in [0,1]]
(x*y + (1-x)*(1-y^2)) >= t
• Consider claims:• C(0.55): Bob strengthens this claim (0.6).• C(0.60): Bob agrees with this claim.• C(0.65): Bob attempts to refute this claim.
3/16/2011 Open House 2011 6
Roles:Alice claimsBob opposes or agrees
Refute
• Alice claimsForAll [x in [0,1]] Exists [y in [0,1]]
(x*y + (1-x)*(1-y^2)) >= 0.65
• Bob refutes• Protocol:
– Bob provides x– Alice provides y– Bob refutes iff (x*y + (1-x)*(1-y^2)) < 0.65
3/16/2011 Open House 2011 7
Karl Popper
3/16/2011 8Open House 2011
SCG is a web-based implementation of Karl Popper’s science ideas
• One of the greatest philosophers of science of the 20th century.
• Falsifiability or refutability is the logical possibility that an assertion could be shown false by a particular observation or physical experiment.
• Error elimination (refutation), performs a similar function for science that natural selection performs for biological evolution.
Open House 2011 9from Wikipedia3/16/2011
Comparison
• Karl Popper: Conjectures and Refutations• Scientific Community Game: Claims and
Refutations• Our claims are about computational
problems.
Open House 2011 103/16/2011
Automating the refutations
• There can be “bugs” in refutations.• With a computational model of scientific
communities we can check for many “bugs”.• Can detect misunderstandings students might
have.• Fair evaluation of scholars.
3/16/2011 11Open House 2011
Designers• SCG• Domain
– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like
• standard solvers and solvers for niches.• providing instances with “interesting” solutions
• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.
• Playground: Configure– Research/Development Managers (Innovation)– Professors (Teaching)
• Avatar– researchers, developers– students
3/16/2011 12Open House 2011
Domain
• Instance (language)• Solution (language)
– boolean valid(Instance)– [0,1] quality(Instance)
• InstanceSet (language, subset of Instance)– boolean valid()– boolean belongsTo(Instance)
• Response = Instance union Solution
3/16/2011 13Open House 2011
SCG(Domain)
• Protocol (fixed language)• Claim(Domain)
– boolean strengthen(Claim other) // other strengthens this
– Domain.InstanceSet getInstanceSet()– Protocol getProtocol()– [0,1] getQuality()– [-1..1] getResult(List(Domain.Response))
3/16/2011 14Open House 2011
Refutations of Claim and !Claim are efficient
Claim: F unsatisfiable• if refuted: Bob finds
satisfying J; proof of !Claim.• If defended: baby step
towards proof of Claim.• Proof: long
!Claim: F satisfiable• if refuted: Alice does not
find satisfying J; baby step towards proof of Claim.
• If defended: proof of !Claim.
• Proof: short
3/16/2011 15Open House 2011
Roles:Alice claimsBob attempts to refute
Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t
• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.
• If defended: baby step towards proof of Claim.
• Proof short.
!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t
• if refuted: Alice does not find J; baby step towards proof of Claim.
• If defended: proof of !Claim if Bob is perfect.
• Proof short.
3/16/2011 16Open House 2011
Claim involving Experiment
Claim ExperimentalTechnique(X,Y,q,r)I claim, given raw materials x in X,I can produce product y in Y of quality qand using resources at most r.
17Bionetics 2010
Our vision
• Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place.
• Participating teams get baby avatars generated for X that participate in daily competitions.
• Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner.
3/16/2011 Open House 2011 18
Conclusions
• Computational Modeling of Scientific Communities is a good idea:– foster Innovation– improve education
• STEM domains: use the web effectively• Current use:
– Algorithms class– Software development class
3/16/2011 Open House 2011 19
3/16/2011 Open House 2011 20
Thankyou!
Thank you!
3/16/2011 Open House 2011 21
Strengthening
• When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold.
• When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation:– Bob + ClaimConfidence + |quality(C)-quality(C')|
• When scholar Alice successfully defends her own claim against Bob, Alice wins reputation.– Alice + ClaimConfidence
3/16/2011 Open House 2011 22
Agreement
• When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob
loses)– (2) Bob must refute C' = C minimally strengthened
along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses).
• Then Alice must do the same:
3/16/2011 Open House 2011 23
Agreement
– (1) Alice must defend C against Bob (if not, Alice loses)
– (2) Alice must refute C' with Bob as defender (if not, Alice loses)
• If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength).
3/16/2011 Open House 2011 24
Refutations of Claim and !Claim are efficient
Claim: F unsatisfiable• if refuted: Bob finds
satisfying J; proof of !Claim.• If defended: baby step
towards proof of Claim.• Proof: long
!Claim: F satisfiable• if refuted: Alice does not
find satisfying J; baby step towards proof of Claim.
• If defended: proof of !Claim.
• Proof: short• Alice should never have
made the claim!
3/16/2011 25Open House 2011
Both refutations are efficientClaim: Exists F in IS All J: fsat(F,J)<=t
• if refuted: Bob finds J; proof of !Claim assuming Alice is perfect.
• If defended: baby step towards proof of Claim.
• Proof short.
!Claim: F has J: fsat(F,J)>=tAll F in IS Exists J: fsat(F,J)>=t
• if refuted: Alice does not find J; baby step towards proof of Claim.
• If defended: proof of !Claim if Bob is perfect.
• Proof short.• Alice should never have
made the claim!?
3/16/2011 26Open House 2011
Designers
• SCG• Domain
– includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver
• Protocol• Playground: Goal: make playground designers
configurators. • Avatar designers
3/16/2011 27Open House 2011
Example Playground DesignHighest Safe Rung
• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 28Open House 2011
Designers: what they produce
• SCG /scg– scg.cd, scg.beh– /protocol
• Java classes: Singleton Pattern
• Domain /domain– /hsr: hsr.cd, hsr.beh
• /avatar (components for avatar)
• Playground– config file: location of configuration file is given as
argument to admin
3/16/2011 29Open House 2011
Config
• Config = // to configure admin<scgCfg> SCGConfig <domainConfigWrapper> Wrap(DomainConfigI).
• Example entries:– domain CSP– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 30Open House 2011
Where can we find configuration settings
• If there is a configuration file location given to the admin– in the configuration file
• If not: the default value given in the code.
3/16/2011 31Open House 2011
3/16/2011 32Open House 2011
Designers• SCG• Domain
– Instance, Solution, InstanceSet, valid, quality– basic domain functionality, like
• standard solvers and solvers for niches.• providing instances with “interesting” solutions
• Protocol: using protocol language– standard protocols: ForAllExists, PositiveSecret, etc.
• Playground: Configurate– Research/Development Managers (Innovation)– Professors (Teaching)
• Avatar– researchers, developers– students
3/16/2011 33Open House 2011
Example Playground DesignHighest Safe Rung
• Configuration:– domain HSR– claim 1: instanceSetClass protocolClass– claim 2: instanceSetClass !protocolClass– initialReputation = 100– …
3/16/2011 34Open House 2011