Specifying revisions, not just branches, in package recipes

[nrvale0]

:branch works with other refs like a tag name. Which is great. But perhaps it should be renamed to :ref?

[raxod502]

This requires some more thought. The repository management algorithms are not actually designed to work with any refs other than branch names, since it's expected that you use version lockfiles to specify exact revisions or tags, rather than putting them in the recipes directly.

[nrvale0]

Ah, ok. I've just gotten started with straight.el so I've not yet explored lockfiles. Entirely possible this is better approach.

[raxod502]

If your goal in specifying a tag or revision in the recipe is that your configuration will use that revision of the package, then indeed version lockfiles are designed to achieve exactly that purpose.

[dieggsy]

That said, it'd be interesting to have a :ref that supports arbitrary refs, with the end goal being that the only source of truth is the init file. This is probably not as intuitive as lockfiles since they're derived from whatever ref you git checkout to, but it might be a nice advanced feature.

[raxod502]

it'd be interesting

Sure, and I wouldn't be opposed to this...

with the end goal being that the only source of truth is the init file

... but I'm honestly not sure why you'd want to do it. You'd have to edit your init-file every time you update a package.

[dieggsy]

Well, if your workflow is one in which you update a lot (which I actually do) then it'd be annoying, but some people might prefer to "pin" packages in that way, esp. if they don't update frequently or want to choose their own definition of "stable enough".

Regardless, it'd be an additional feature, so you wouldn't be required to 'pin' every package with a specific ref throughout. But if a package came up that you did want to pin or revert, you'd have the convenience of simply specifying the ref in the init file, without pinning everything to its current state through straight-freeze-versions

[nrvale0]

... but I'm honestly not sure why you'd want to do it. You'd have to edit your init-file every time you update a package.

I can speak to this a bit. I went down the path of straight.el to allow me to install the latest version of Org and associated packages without conflicts with RPM-packaged Org on Fedora (not possible with use-package). All other packages are still installed with use-package.

So, in my case, having a lockfile separate from the package specification in my init-file is just one more thing to track in the repo and in my head with not much payout.

Though I agree, having worked with similar setups in other environments (requirements.txt, Gemfile.lock, etc) that specification in the init-file probably does not scale beyond a handful of packages under straight.el management.

[raxod502]

Regardless, it'd be an additional feature

Fair enough, I see no reason why this can't coexist with the current functionality.

that specification in the init-file probably does not scale

Indeed, especially once you consider that not only do you have to add a revision manually for every package that is mentioned in your init-file, but also all of their dependencies (which can only be determined by looking at the source code of those packages), as well as any dependencies that might be added when you update a package. This would need tooling to be manageable (not that there is anything wrong with that).

[mnewt]

I have a use case, which is similar to those discussed but perhaps with a little different nuance. I usually update all my packages at once, with these commands:

straight-normalize-all
straight-fetch-all
straight-merge-all
;; verify that things are working
straight-freeze-versions

For most packages, that works great. However, I have a couple packages I don't want to update, but instead pin at a known good release or commit. So, for most packages, I'm happy to write a straight/versions/default.el file out without having to micro-manage each revision but would like to only explicitly pin certain packages with :ref or similar.

Would the current way be to use the profile system?

This proposal seems the best way forward to me.

[raxod502]

The interaction of the profile system and lockfiles needs some more work, see e.g. #67. Currently there isn't really any interaction.

I agree that it would be nice if we could support your use case using an existing feature. Some thought would be required to determine if this is possible. I don't necessarily have an objection to implementing a separate pinning system, as long as the implications are well-thought-out.

As a workaround, you could simply write your own version of straight-merge-all which skips packages in your list:

(defun my-straight-merge-unpinned (&optional from-upstream predicate)
  "Try to merge all packages from their primary remotes.
With prefix argument FROM-UPSTREAM, merge not just from primary
remotes but also from configured upstreams.

Return a list of recipes for packages that were not successfully
merged. If multiple packages come from the same local
repository, only one is merged.

PREDICATE, if provided, filters the packages that are merged. It
is called with the package name as a string, and should return
non-nil if the package should actually be merged.

PREDICATE is automatically modified so that packages in the list
`my-pinned-packages' are not merged."
  (interactive "P")
  (straight--map-existing-repos-interactively
   (lambda (package)
     (straight-merge-package package from-upstream))
   (lambda (package)
     (and (not (member package my-pinned-packages))
          (or (null predicate) (funcall predicate package))))))

Warning: I have not tested this code.

[mnewt]

@raxod502 I just tested your code and it works perfectly. When I run the straight-*-all and straight-freeze-versions commands, the packages in the my-pinned-packages list are not updated. This is a good workaround and anyone else with this use case can simply copy and paste.

Many thanks!

[raxod502]

It's not clear why I failed to think of this earlier, but you can easily do this without duplicating the logic of straight-merge-all and without using internal functions:

(defun my-straight-merge-all (&optional from-upstream)
  "Try to merge all packages from their primary remotes.
With prefix argument FROM-UPSTREAM, merge not just from primary
remotes but also from configured upstreams.

Do not merge packages listed in `my-pinned-packages'."
  (interactive "P")
  (straight-merge-all
   from-upstream
   (lambda (package)
     (not (member package my-pinned-packages)))))