a new, modular dependency solver for...
TRANSCRIPT
![Page 1: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/1.jpg)
A new, modulardependency solver for cabal-install
Andres LohWell-Typed LLP
Duncan CouttsWell-Typed LLP
HIW 2011
23 September 2011
![Page 2: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/2.jpg)
Why?
I Error messages are suboptimal.I We want the solver to be more configurable.I Solver is difficult to extend (current design is rather
ad-hoc).
![Page 3: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/3.jpg)
Why now?
We have started to work on this as part of a project funded bythe Industrial Haskell Group.
Original trigger: support “private dependencies”.
Status
I New solver is implemented.I “At least as good” as the old solver
– when used with ghc-7.*
I Ready for experimentation.I Some new features prepared, but not yet implemented.
![Page 4: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/4.jpg)
Why now?
We have started to work on this as part of a project funded bythe Industrial Haskell Group.
Original trigger: support “private dependencies”.
Status
I New solver is implemented.I “At least as good” as the old solver
– when used with ghc-7.*
I Ready for experimentation.I Some new features prepared, but not yet implemented.
![Page 5: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/5.jpg)
Talk overview
I Dependencies in CabalI Architecture of the new solverI Private dependenciesI Error messagesI Remaining work
![Page 6: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/6.jpg)
Dependencies in Cabal
![Page 7: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/7.jpg)
cabal-install internals
When you call cabal install,
1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 8: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/8.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,
3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 9: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/9.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,
4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 10: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/10.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,
5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 11: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/11.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 12: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/12.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 13: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/13.jpg)
cabal-install internals
When you call cabal install,1. the Hackage index is read,2. the installed packages index is read,3. the solver is invoked to produce an install plan,4. the install plan is sanity checked,5. the install plan is executed (or printed).
Independent sanity check makes it relatively easy to trust thenew solver.
![Page 14: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/14.jpg)
Terminology
Index database of information aboutpackages (can be built from manyindices)
Location an index location (such as Hackage,or your installed packages)
Package Name a name such as mtl or threadscopeVersion a version such as 1.1.0.2
Instance a triple of a name, version andlocation
Instance Constraint restrictions on version and locationDependency a pair of a name and a constraint
![Page 15: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/15.jpg)
Cabal dependency problem
I Cabal files can specify dependencies based on booleanflags.
I The complete transitive dependency closure of a packageis needed to uniquely identify it (ABI hash).
I In other words: installed instances have fixeddependencies, new instances have flexible dependencies.
I In general, one application cannot use multiple instancesof the same package.
I With ghc-pkg, we can install many instances of onepackage, but only one instance per package version.
![Page 16: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/16.jpg)
Cabal dependency problem
I Cabal files can specify dependencies based on booleanflags.
I The complete transitive dependency closure of a packageis needed to uniquely identify it (ABI hash).
I In other words: installed instances have fixeddependencies, new instances have flexible dependencies.
I In general, one application cannot use multiple instancesof the same package.
I With ghc-pkg, we can install many instances of onepackage, but only one instance per package version.
![Page 17: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/17.jpg)
Cabal dependency problem
I Cabal files can specify dependencies based on booleanflags.
I The complete transitive dependency closure of a packageis needed to uniquely identify it (ABI hash).
I In other words: installed instances have fixeddependencies, new instances have flexible dependencies.
I In general, one application cannot use multiple instancesof the same package.
I With ghc-pkg, we can install many instances of onepackage, but only one instance per package version.
![Page 18: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/18.jpg)
Cabal dependency problem
I Cabal files can specify dependencies based on booleanflags.
I The complete transitive dependency closure of a packageis needed to uniquely identify it (ABI hash).
I In other words: installed instances have fixeddependencies, new instances have flexible dependencies.
I In general, one application cannot use multiple instancesof the same package.
I With ghc-pkg, we can install many instances of onepackage, but only one instance per package version.
![Page 19: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/19.jpg)
Cabal dependency problem
I Cabal files can specify dependencies based on booleanflags.
I The complete transitive dependency closure of a packageis needed to uniquely identify it (ABI hash).
I In other words: installed instances have fixeddependencies, new instances have flexible dependencies.
I In general, one application cannot use multiple instancesof the same package.
I With ghc-pkg, we can install many instances of onepackage, but only one instance per package version.
![Page 20: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/20.jpg)
Approach of the old solver
Conservative approach
Tries to select a unique instance per package in an install plan.
Furthermore:I Never backtracks (but looks ahead a bit).I Exclude packages that can’t be configured (relatively new).I Flag resolution tied to package selection.I Maintains hardly any information about the order in which
packages are resolved.
![Page 21: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/21.jpg)
Approach of the old solver
Conservative approach
Tries to select a unique instance per package in an install plan.
Furthermore:I Never backtracks (but looks ahead a bit).I Exclude packages that can’t be configured (relatively new).I Flag resolution tied to package selection.I Maintains hardly any information about the order in which
packages are resolved.
![Page 22: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/22.jpg)
Cabal can break your system
B-1
A
Assume B-1 depends on any version of A.
![Page 23: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/23.jpg)
Cabal can break your system
B-1
A-1
You install B-1 on your system, fixing the dependency to A-1.
![Page 24: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/24.jpg)
Cabal can break your system
B-1
A-1
C-1. . .
Many other packages that depend on B-1 are installed later.
![Page 25: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/25.jpg)
Cabal can break your system
B-1
A-1
C-1. . .
B-1
A-2
D-1
Now we want to install D which depends on A-2 (!) and B.
![Page 26: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/26.jpg)
Cabal can break your system
B-1
A-1
C-1. . .
B-1
A-2
D-1
Since B still depends on A, the install plan selects A-2.
![Page 27: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/27.jpg)
Cabal can break your system
A-1
C-1. . .
B-1
A-2
D-1
Upon actual installation, the old B-1 is destructively updated . . .
![Page 28: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/28.jpg)
About destructive updates
Proper solution
NixOS-style package database – no destructive updates, ever.
Other options
I warn explicitly about destructive updatesI discourage or prevent destructive updates in the solver
options
![Page 29: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/29.jpg)
About destructive updates
Proper solution
NixOS-style package database – no destructive updates, ever.
Other options
I warn explicitly about destructive updatesI discourage or prevent destructive updates in the solver
options
![Page 30: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/30.jpg)
The new solver
![Page 31: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/31.jpg)
The algorithm
1. Build (lazily) an approximation of the search tree.The tree contains all valid solutions (but may contain wrongones).
2. Validate the search tree.The resulting tree contains only valid solutions.
3. Rearrange and optimize the search tree.Many independent traversals over the tree. This is whatmakes the solver modular.
4. Explore the search tree.Once a solution is found, we turn it into an install plan.
Inspired by . . .
Thomas Nordin and Andrew Tolmach, Modular Lazy Search forConstraint Satisfaction Problems, JFP, 2001
![Page 32: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/32.jpg)
The algorithm
1. Build (lazily) an approximation of the search tree.The tree contains all valid solutions (but may contain wrongones).
2. Validate the search tree.The resulting tree contains only valid solutions.
3. Rearrange and optimize the search tree.Many independent traversals over the tree. This is whatmakes the solver modular.
4. Explore the search tree.Once a solution is found, we turn it into an install plan.
Inspired by . . .
Thomas Nordin and Andrew Tolmach, Modular Lazy Search forConstraint Satisfaction Problems, JFP, 2001
![Page 33: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/33.jpg)
The algorithm
1. Build (lazily) an approximation of the search tree.The tree contains all valid solutions (but may contain wrongones).
2. Validate the search tree.The resulting tree contains only valid solutions.
3. Rearrange and optimize the search tree.Many independent traversals over the tree. This is whatmakes the solver modular.
4. Explore the search tree.Once a solution is found, we turn it into an install plan.
Inspired by . . .
Thomas Nordin and Andrew Tolmach, Modular Lazy Search forConstraint Satisfaction Problems, JFP, 2001
![Page 34: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/34.jpg)
The algorithm
1. Build (lazily) an approximation of the search tree.The tree contains all valid solutions (but may contain wrongones).
2. Validate the search tree.The resulting tree contains only valid solutions.
3. Rearrange and optimize the search tree.Many independent traversals over the tree. This is whatmakes the solver modular.
4. Explore the search tree.Once a solution is found, we turn it into an install plan.
Inspired by . . .
Thomas Nordin and Andrew Tolmach, Modular Lazy Search forConstraint Satisfaction Problems, JFP, 2001
![Page 35: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/35.jpg)
The algorithm
1. Build (lazily) an approximation of the search tree.The tree contains all valid solutions (but may contain wrongones).
2. Validate the search tree.The resulting tree contains only valid solutions.
3. Rearrange and optimize the search tree.Many independent traversals over the tree. This is whatmakes the solver modular.
4. Explore the search tree.Once a solution is found, we turn it into an install plan.
Inspired by . . .
Thomas Nordin and Andrew Tolmach, Modular Lazy Search forConstraint Satisfaction Problems, JFP, 2001
![Page 36: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/36.jpg)
The algorithm
solve cfg idx userPrefs userConstraints userGoals =explorePhase $heuristicsPhase $preferencesPhase $validationPhase $prunePhase $buildPhasewhere
buildPhase :: Tree . . .prunePhase :: Tree . . . → Tree . . .. . .explorePhase :: Tree . . . → Log (Assignment,RevDepMap)
![Page 37: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/37.jpg)
The search tree
data Tree a =PChoice QPN a (PSQ I (Tree a))
| FChoice QFN a Bool (PSQ Bool (Tree a))| GoalChoice (PSQ OpenGoal (Tree a))| Done RevDepMap| Fail (ConflictSet QPN) FailReason
I Nodes represent goals / choices.I Edges represent assignments.I The search tree is not the dependency tree.
![Page 38: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/38.jpg)
The search tree
data Tree a =PChoice QPN a (PSQ I (Tree a))
| FChoice QFN a Bool (PSQ Bool (Tree a))| GoalChoice (PSQ OpenGoal (Tree a))| Done RevDepMap| Fail (ConflictSet QPN) FailReason
I Nodes represent goals / choices.I Edges represent assignments.I The search tree is not the dependency tree.
![Page 39: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/39.jpg)
The search tree
Several kinds of nodes:Goal Choice to give some flexibility to the order in
which goals are solved
Package Goals to pick an instance for a given nameFlag Goals to select a boolean for a given flag for
a given packageSuccess to indicate that a solution has been
foundFail to explicitly indicate failure
All nodes are equipped with additional info.
Note that goal-choice nodes have a different semantics fromsingle-goal nodes.
![Page 40: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/40.jpg)
The search tree
Several kinds of nodes:Goal Choice to give some flexibility to the order in
which goals are solvedPackage Goals to pick an instance for a given name
Flag Goals to select a boolean for a given flag fora given package
Success to indicate that a solution has beenfound
Fail to explicitly indicate failure
All nodes are equipped with additional info.
Note that goal-choice nodes have a different semantics fromsingle-goal nodes.
![Page 41: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/41.jpg)
The search tree
Several kinds of nodes:Goal Choice to give some flexibility to the order in
which goals are solvedPackage Goals to pick an instance for a given name
Flag Goals to select a boolean for a given flag fora given package
Success to indicate that a solution has beenfound
Fail to explicitly indicate failure
All nodes are equipped with additional info.
Note that goal-choice nodes have a different semantics fromsingle-goal nodes.
![Page 42: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/42.jpg)
The search tree
Several kinds of nodes:Goal Choice to give some flexibility to the order in
which goals are solvedPackage Goals to pick an instance for a given name
Flag Goals to select a boolean for a given flag fora given package
Success to indicate that a solution has beenfound
Fail to explicitly indicate failure
All nodes are equipped with additional info.
Note that goal-choice nodes have a different semantics fromsingle-goal nodes.
![Page 43: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/43.jpg)
Building the tree
Keep track of current goals:I build goal choice nodeI build goal-specific nodesI add new goals depending on choice if neededI if no goals left, end with a success node
![Page 44: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/44.jpg)
Validating the tree
Keep track of constraints:I use constraints to remove choicesI but keep disabled choices around (for error messages)I add new constraints corresponding to the choices while
moving downAfter validation, the tree contains just correct solutions.
![Page 45: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/45.jpg)
Reorderings
I exploration will in essence proceed depth-first, left-to-rightI the order of subtrees in the choice nodes is relevantI we can thus express preferences by reordering
![Page 46: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/46.jpg)
Typical preferences
I Prefer later versions over older versions.I Prefer installed instances over non-installed ones.
packageOrderFor :: (PN → Bool) →(PN → I → I → Ordering) →Tree a → Tree a
packageOrderFor p cmp = trav gowhere
go (PChoiceF v@(Q pn) r cs)| p pn = PChoiceF v r
(sortByKeys (flip (cmp pn)) cs)| otherwise = PChoiceF v r cs
go x = x
![Page 47: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/47.jpg)
Typical preferences
I Prefer later versions over older versions.I Prefer installed instances over non-installed ones.
packageOrderFor :: (PN → Bool) →(PN → I → I → Ordering) →Tree a → Tree a
packageOrderFor p cmp = trav gowhere
go (PChoiceF v@(Q pn) r cs)| p pn = PChoiceF v r
(sortByKeys (flip (cmp pn)) cs)| otherwise = PChoiceF v r cs
go x = x
![Page 48: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/48.jpg)
Goal heuristics
Goal choices allow us to do some limited look-ahead:I Prefer goals that lead to immediate failure.I Prefer goals (package names and flags) that have only one
allowed choice.
Because it does never make sense to backtrack in goal-choicenodes, we actually leave only the first goal choice after applyinggoal heuristics.
![Page 49: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/49.jpg)
Goal heuristics
Goal choices allow us to do some limited look-ahead:I Prefer goals that lead to immediate failure.I Prefer goals (package names and flags) that have only one
allowed choice.
Because it does never make sense to backtrack in goal-choicenodes, we actually leave only the first goal choice after applyinggoal heuristics.
![Page 50: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/50.jpg)
Backjumping
We prune the tree by propagating failure information up.
Key observation
Not all the nodes on a path to a conflict actually contribute tothat conflict.
So in many cases we can prune entire failing subtrees quickly.
We also use conflict set info for error messages.
![Page 51: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/51.jpg)
Backjumping
We prune the tree by propagating failure information up.
Key observation
Not all the nodes on a path to a conflict actually contribute tothat conflict.
So in many cases we can prune entire failing subtrees quickly.
We also use conflict set info for error messages.
![Page 52: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/52.jpg)
Backjumping
We prune the tree by propagating failure information up.
Key observation
Not all the nodes on a path to a conflict actually contribute tothat conflict.
So in many cases we can prune entire failing subtrees quickly.
We also use conflict set info for error messages.
![Page 53: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/53.jpg)
Exploring the tree
Mainly depth-first, left-to-right being used, but with variousdegrees of debugging info being produced.
Configurable backtracking: we can impose a limit on thenumber of backjumps performed – the old solver neverbacktracks.
![Page 54: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/54.jpg)
Exploring the tree
Mainly depth-first, left-to-right being used, but with variousdegrees of debugging info being produced.
Configurable backtracking: we can impose a limit on thenumber of backjumps performed – the old solver neverbacktracks.
![Page 55: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/55.jpg)
Reflections on the implementation
I The approach is reasonably easy to work with in practice,because we can split the algorithm into multipleindependent steps.
I We haven’t made any effort on clever optimizations, andthe new solver has about the same speed in practice asthe old one . . .
I However, while laziness is the key to modularity here, thereare also very subtle laziness constraints that aren’texpressible in the type system.
![Page 56: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/56.jpg)
Encapsulations
![Page 57: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/57.jpg)
Encapsulations
The assumption to allow only one instance of a package perapplication is too conservative:
I The problem arises from exported types that aren’tcompatible between instances.
I Some libraries are mostly used privately or don’t exportany types.
I If we could declare such private dependencies, we mightget better results.
Idea
We allow a package A to declare that it encapsulates packageB. Ideally, it should be checked that A’s interface contains notraces of B.
However, encapsulations are subtle . . .
![Page 58: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/58.jpg)
Encapsulations
The assumption to allow only one instance of a package perapplication is too conservative:
I The problem arises from exported types that aren’tcompatible between instances.
I Some libraries are mostly used privately or don’t exportany types.
I If we could declare such private dependencies, we mightget better results.
Idea
We allow a package A to declare that it encapsulates packageB. Ideally, it should be checked that A’s interface contains notraces of B.
However, encapsulations are subtle . . .
![Page 59: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/59.jpg)
Encapsulations
The assumption to allow only one instance of a package perapplication is too conservative:
I The problem arises from exported types that aren’tcompatible between instances.
I Some libraries are mostly used privately or don’t exportany types.
I If we could declare such private dependencies, we mightget better results.
Idea
We allow a package A to declare that it encapsulates packageB. Ideally, it should be checked that A’s interface contains notraces of B.
However, encapsulations are subtle . . .
![Page 60: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/60.jpg)
Encapsulation example
A
B
C
C.B
Different instances can be chosen for B and C.B.
![Page 61: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/61.jpg)
Encapsulation example
A
B
D
C
C.B
The encapsulation is invalidated by other dependencies. Both Band C.B must be the same instance.
![Page 62: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/62.jpg)
The plan for encapsulations
I The solver is prepared for scoped goals.I Scoped goals are introduced when packages declare
encapsulations.I Scoped goals can be invalidated by other dependencies
(i.e., forced to be equal to original goals).I Scoped goals should therefore be resolved as late as
possible, to prevent unnecessary backtracking.I Even if scoped goals can be resolved differently, the
preference should still be to select a single instance perpackage.
I The install plan checker has to be generalized.I The Cabal library needs to be extended to check private
dependencies.
![Page 63: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/63.jpg)
Error messages
Ideas
I Add as much information internally as possible, i.e., keepreasons for all decisions.
I Make traces, while being verbose, fully understandable.I Error messages are excerpts of the full trace, removing
irrelevant parts.
Current status
I All the information is there.I Could be presented in a better/cleaner way.I There are some choices on how to produce excerpts.
![Page 64: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/64.jpg)
Error messages
Ideas
I Add as much information internally as possible, i.e., keepreasons for all decisions.
I Make traces, while being verbose, fully understandable.I Error messages are excerpts of the full trace, removing
irrelevant parts.
Current status
I All the information is there.I Could be presented in a better/cleaner way.I There are some choices on how to produce excerpts.
![Page 65: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/65.jpg)
Demo
![Page 66: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/66.jpg)
Future work
I Turn cabal-install into a proper library.I Factor out the solver into a separate package.I Others can write their own solvers.I Integrate cabal-dev, or make such tools use the library.I Track external dependencies.I Allow more configuration options.
![Page 67: A new, modular dependency solver for cabal-installwiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdfcabal-install internals When you call cabal install, 1. the Hackage index is](https://reader031.vdocument.in/reader031/viewer/2022040711/5e12ece4fd3b0150963118db/html5/thumbnails/67.jpg)
Please try it
We appreciate early feedback:
darcs get \http://darcs.haskell.org/cabal-branches/cabal-modular-solver
cd cabal-modular-solvercabal install ./cabal ./cabal-install