[RFC] Cabal plugin infrastructure

[Kleidukos]

This is a WIP. Fendor & I will edit this description to refine the features & requirements.

Inspirations

• git
• cargo

Reasoning

To avoid bloat in cabal-install CLI and offer a more ergonomic way to use plugins, we must offer first-class support for cabal CLI plugins.

• cabal why-depends (Implement why-depends command haskell-hvr/cabal-plan#53)
• cabal status (Add status command to cabal #7500)

More importantly, it allows us to provide features in "beta" or "experimental" status to our users before integrating them into the CLI codebase when they are more mature.
This is a low-tech solution but tech is not the only aspect being considered here.

What we need

• Plugin infrastructure for cabal CLI

  • cabal instal --plugin
    • It is installed in ~/.cabal/bin XOR ~/.cabal/plugins/bin
    • When people type cabal foo, if foo is not a valid subcommand, we lookup cabal-foo in the bin directory.
  • Provide a cabal-metadata package as companion tooling for plugin authors
    • Provide structure representation of the plan.json & build-info.json
  • Handle incompatibilities between Cabal versions. Need more thought.
    • JSON incompatibilities
    • Binary incompatibilities

• Cabal version compatibility

  • One plugin version per project?
  • One plugin version per installed ghc.
    • Call ghcup directly to install cabal plugins. Symlink logic is reused, the same as HLS

• Cabal command-line arguments passing

  • This amounts to calling another binary with the parameters passed. Do we want to setup some stuff on the cabal-install side (like environment variables) before calling the binary (cabal-foo in ~/.cabal/bin)?

cc @fendor

[jneira]

Having that would be really great, thanks for the proposal

Related: #2349, #2395

[gbaz]

I have a fair amount of trepidation about this, and my main comment would be that we should follow the model of git and cargo, which seems to be proposal #2349 and keep this as simple as possible. I propose that we do not need to have a --plugin flag for cabal install. It just installs plugins like any other binaries. It just looks up executables by name to execute subcommands, and further, perhaps, lists them under cabal --help. I also suggest we do nothing directly to enforce version compat, and we do nothing to worry about cli or env argument passing. I.e. this is just sugar for invoking a subcommand.

Anything else, while feeling "clever" at first, will in my estimation lead to more confusing and undocumented behavior in the long run.

[Kleidukos]

@gbaz thanks for restricting the scope :)

[Ericson2314]

I don't think cabal-foo vs cabal foo is a very high priority thing to worry about. I have used Cargo plugins and git plugins and don't really see the point of that feature over there either.

However, trying to break out some libraries from Cabal and cabal-install with just parsers and ASTs is always a good idea, and one I am perennially excited about :). (We should do the same thing with GHC too, so ghc depends on ghc-lib-parser or so!)

[Kleidukos]

I don't think cabal-foo vs cabal foo is a very high priority thing to worry about. I have used Cargo plugins and git plugins and don't really see the point of that feature over there either.

@Ericson2314 They give better ergonomy and first-class support that we can improve later on. We can also organise a "beta" workflow of new subcommands if they can live in separate binaries during their "experimental" life cycle, before being integrated in the codebase of the CLI.

Thank you for your support regarding cabal-metadata. :)

[jneira]

They give better ergonomy and first-class support that we can improve later on. We can also organise a "beta" workflow of new subcommands if they can live in separate binaries during their "experimental" life cycle, before being integrated in the codebase of the CLI.

That is a key argument, we could, for example, bring here cabal-plan and have cabal plan almost for free

[Kleidukos]

@jneira The idea was to implement the featureset described here as a plugin: #7500 (comment)

[yvan-sraka]

After reading through the discussions here (and discussing it while ZuriHac with @Kleidukos), I propose to stick to a simple, straightforward approach for a cabal-install CLI plugin system, inspired by how git and cargo handle commands:

1. Treat each plugin as a standalone executable named cabal-$COMMAND in the PATH. It should be callable as cabal $COMMAND ;
2. Pass any additional CLI arguments directly to this command. This allows plugins to range from a simple shell script to more complex applications ;
3. Rather than, e.g., passing data in specific formats like JSON or constraining the user to link their program against the Cabal library, we provide necessary information (like GHC version or project root directory) as ENV variables. This allows plugins to rely on cabal-install CLI or Cabal library as they wish, or not at all. cargo follows a similar approach: https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-3rd-party-subcommands

I've taken a brief look at the Cabal codebase, and it seems feasible to implement this approach just by slightly modifying this function:

cabal/Cabal/src/Distribution/Simple/Command.hs

Lines 622 to 687 in 1cb92a3

	commandsRun
	:: CommandUI a
	-> [Command action]
	-> [String]
	-> CommandParse (a, CommandParse action)
	commandsRun globalCommand commands args =
	case commandParseArgs globalCommand True args of
	CommandHelp help -> CommandHelp help
	CommandList opts -> CommandList (opts ++ commandNames)
	CommandErrors errs -> CommandErrors errs
	CommandReadyToGo (mkflags, args') -> case args' of
	("help" : cmdArgs) -> handleHelpCommand cmdArgs
	(name : cmdArgs) -> case lookupCommand name of
	[Command _ _ action _] ->
	CommandReadyToGo (flags, action cmdArgs)
	_ -> CommandReadyToGo (flags, badCommand name)
	[] -> CommandReadyToGo (flags, noCommand)
	where
	flags = mkflags (commandDefaultFlags globalCommand)
	where
	lookupCommand cname =
	[ cmd | cmd@(Command cname' _ _ _) <- commands', cname' == cname
	]
	noCommand = CommandErrors ["no command given (try --help)\n"]
	-- Print suggested command if edit distance is < 5
	badCommand :: String -> CommandParse a
	badCommand cname =
	case eDists of
	[] -> CommandErrors [unErr]
	(s : _) ->
	CommandErrors
	[ unErr
	, "Maybe you meant `" ++ s ++ "`?\n"
	]
	where
	eDists =
	map fst . List.sortBy (comparing snd) $
	[ (cname', dist)
	| (Command cname' _ _ _) <- commands'
	, let dist = editDistance cname' cname
	, dist < 5
	]
	unErr = "unrecognised command: " ++ cname ++ " (try --help)"
	commands' = commands ++ [commandAddAction helpCommandUI undefined]
	commandNames = [name | (Command name _ _ NormalCommand) <- commands']
	-- A bit of a hack: support "prog help" as a synonym of "prog --help"
	-- furthermore, support "prog help command" as "prog command --help"
	handleHelpCommand cmdArgs =
	case commandParseArgs helpCommandUI True cmdArgs of
	CommandHelp help -> CommandHelp help
	CommandList list -> CommandList (list ++ commandNames)
	CommandErrors _ -> CommandHelp globalHelp
	CommandReadyToGo (_, []) -> CommandHelp globalHelp
	CommandReadyToGo (_, (name : cmdArgs')) ->
	case lookupCommand name of
	[Command _ _ action _] ->
	case action ("--help" : cmdArgs') of
	CommandHelp help -> CommandHelp help
	CommandList _ -> CommandList []
	_ -> CommandHelp globalHelp
	_ -> badCommand name
	where
	globalHelp = commandHelp globalCommand

This method keeps the system simple and functional, gives users more control over plugin creation and installation, and allows for a “beta” workflow for new commands before they're integrated into the CLI codebase. WDYT @fendor?

[fendor]

Hi!

After discussing the proposed solution with @Kleidukos, this looks like a good way forward! I don't have much to add for now, looking forward to the PR!

[Kleidukos]

I have a question: should we encourage the usage of extra field (starting with x-) in cabal.project? I'm writing a cabal plugin that would read from the codebase, and obviously the maintainers of a project may want to exclude some paths, and specifying file patterns.

@gbaz This question is for you: Was there ever a plan to support arbitrary stanzas that operate the same as source-repository-package (that is, with subfields)? If they started with x- (like in the .cabal format), cabal-install could treat them as opaque and the tools would bring their own parser for the file.

I'm especially looking at

cabal/cabal-install/src/Distribution/Client/Types/SourceRepo.hs

Lines 101 to 118 in 1264967

	sourceRepositoryPackageGrammar
	:: ( FieldGrammar c g
	, Applicative (g SourceRepoList)
	, c (Identity RepoType)
	, c (List NoCommaFSep FilePathNT String)
	, c (NonEmpty' NoCommaFSep Token String)
	)
	=> g SourceRepoList SourceRepoList
	sourceRepositoryPackageGrammar =
	SourceRepositoryPackage
	<$> uniqueField "type" srpTypeLens
	<*> uniqueFieldAla "location" Token srpLocationLens
	<*> optionalFieldAla "tag" Token srpTagLens
	<*> optionalFieldAla "branch" Token srpBranchLens
	<*> monoidalFieldAla "subdir" (alaList' NoCommaFSep FilePathNT) srpSubdirLens -- note: NoCommaFSep is somewhat important for roundtrip, as "." is there...
	<*> fmap (maybe [] toList) pcc
	where
	pcc = optionalFieldAla "post-checkout-command" (alaNonEmpty' NoCommaFSep Token) srpCommandLensNE

and I wonder if the plugins couldn't define their own (since most of the stuff is defined in Cabal-syntax).

However the parser itself is defined in cabal-install, so that would mean taking it out? I'm not super confident that it is a painless process.