21 Sep 2015
We recently had a very large discussion of stack on Reddit, which I thought was great for kicking off some discussion. In that discussion, there was a very active thread about how stack relates to the Package Versioning Policy (aka, the PVP).
The PVP - and in particular its policy on preemptive upper bounds - has been discussed at great length many times in the past, and I have no wish to revisit that discussion here. I also went into some detail on Reddit explaining how stack and Stackage relate to the PVP, and won't repeat those details here (tl;dr: they're orthogonal issues, and upper bounds are neither required nor discouraged by either).
However, the discussion brought up one of my long-held beliefs: "the right way to solve this is with tooling." Specifically: manually keeping track of lots of lower and upper bounds is a hard, tedious process that many people get wrong regularly. One great effort in this direction is the Hackage Matrix Builder project, which helps find overly lax (and, I believe, overly restrictive) bounds on Hackage and report them to authors. I'm announcing an experimental feature I've just added to stack master, and hoping it can help with the initial creation of upper bounds.
The feature itself is quite simple. When you run the
upload commands, there's a new option:
--pvp-bounds, which can be
none (don't modify the cabal file at all),
upper (add upper
lower (add lower bounds), and
both (add both upper and
lower bounds). The default is
none (we shouldn't change an author's
cabal file without permission), but that default can be overridden in
stack.yaml file (either for your project, or in
~/.stack/stack.yaml). The algorithm is this:
- Find every single dependency in the .cabal file (e.g.,
bytestring >= 0.9)
- If the user has asked for lower bounds to be added, and that dependency has no lower bound, set the lower bound to the package version specified in the stack.yaml file
- If the user has asked for upper bounds to be added, and that dependency has no upper bound, set the upper bound to be less than the next major version of the package relative to what's specified in the stack.yaml file
That was a bit abstract, so let's give an example. Support you're using LTS 3.0, which includes aeson-0.8.0.2, attoparsec-0.12.1.6, and text-184.108.40.206. Let's further say that in your cabal file you have the following:
build-depends: aeson >= 0.7 , text < 2 , attoparsec
If you specify
--pvp-bounds both on the command line, you'll end up
with the following changes:
- aeson will now be specified as
aeson >= 0.7 && < 0.9. Reason: We respect the existing lower bound (>= 0.7), but add in an upper bound based on the version of aeson used in your snapshot. Since we're currently using 0.8.0.2, the next major bump will be 0.9.
- text will be
text >= 220.127.116.11 && < 2. We respect the existing upper bound (even though it's no in compliance with PVP rules - this allows you as an author to maintain more control when using this feature), but add in a lower bound to the currently used version of text.
- attoparsec will be
attoparsec >= 0.12.1.6 && < 0.13. Since there are no upper or lower bounds, we add both of them in.
Just to round out the feature description:
- if you use
--pvp-bounds none, your bounds are unmodified
--pvp-bounds loweryou get
- aeson >= 0.7
- text >= 18.104.22.168 && < 2
- attoparsec >= 0.12.1.6
--pvp-bounds upperyou get
- aeson >= 0.7 && < 0.9
- text < 2
- attoparsec < 0.13
The motivation behind this approach is simplicity. For users of stack, your versioning work usually comes down to choosing just one number: the LTS Haskell version (or Stackage Nightly date), which is relatively easy to deal with. Managing version ranges of every single dependency is an arduous process, and hard to get right. (How often have you accidentally started relying on a new feature in a minor version bump of a dependency, but forgotten to bump the lower bound?) Now, stack will handle that drudgery for you.
Of course, there are cases where you'll want to tell stack that you know better than it, e.g. "I'm using a subset of the aeson API that's compatible between both 0.7 and 0.8, so I want to override stack's guess at a lower bound." Or, "even though the PVP says text-1.3 may have a breaking change, I've spoken with the author, and I know that the parts I'm using won't be changed until version 2."
This feature should still be considered experimental, but I'm hopeful that it will be an experiment that works, and can make both upper bounds advocates and detractors happy. As a member of the latter group, I'm actually planning on trying out this functionality on some of my packages for future releases.
There are still downsides to this feature, which are worth mentioning:
- Lower bounds may be too restrictive by default. Solution: consider
- You'll still have to manually relax upper bounds on Hackage. Solution: make sure your project is in Stackage so you get early notifications, and add relaxed upper bounds manually to your .cabal file as necessary.
- This does nothing to add upper bounds to existing uploads to Hackage. Maybe someone can add a tool to do that automatically for you (or, even better, enhance cabal's dependency solver to take date information into account)
- In the past, a strong reason to avoid upper bounds was that it would
trigger bugs in the dependency solver that could prevent a valid
build plan from being found. For the most part, those bugs have been
resolved, but on occasion you may still need to use the
--reorder-goals --max-backjumps=-1flag to cabal. (Note: when using stack's dependency solving capabilities, it passes in these flags for you automatically.)