Skip to content

Reintroduce hls-retrie-plugin - #5147

Open
simonhorlick wants to merge 112 commits into
haskell:masterfrom
simonhorlick:hls-retrie3
Open

simonhorlick wants to merge 112 commits into
haskell:masterfrom
simonhorlick:hls-retrie3

Conversation

@simonhorlick

@simonhorlick simonhorlick commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

This PR resurrects the hls-retrie-plugin.

The previous retrie plugin had a few issues that I wanted to ensure were solved by the new plugin:

  • imports were often missing or incorrect
  • grafting often introduced syntax errors
  • precedence was not handled correctly
  • test coverage was quite sparse
  • several correctness issues have also been fixed upstream

This is the result of many attempts and failures, each leading to a hopefully better implementation. I have tried to document below the known shortcomings, many of these I intend to address with follow up PRs.

AI disclaimer: Some of the test cases were suggested by Claude. The plugin code itself is almost completely hand written.

What's supported

  • Inlining of top-level definitions, infix operators and sections
  • Inlining of let and where bindings
  • Patterns in the definitions arguments
  • Bindings in the expression are renamed to avoid capture
  • Where bindings in the definition are rewritten as let bindings in the rewrite
  • Use sites that are not fully applied get lambdas inserted
  • Cross-module support (including adding of imports)

What's missing?

  • Definitions with several clauses, guards or a record wildcard construction. Previously these were handled incorrectly, now they are refused
  • We do not emit changes for non-active CPP branches
  • Visible type application
  • Recursive functions
  • Functions that reference a type variable in the expression
  • Cross-module inlining where the target module is missing a language pragma
  • Type constraints solved by the expressions signature are not propagated to the inlining site
  • Duplication of arguments (inlining could cause an argument to be evaluated multiple times)
  • Capture avoidance outside of the substitution (if a binder somewhere on the path to the splice point captures a free variable in the template)
  • Rewrites into CPP modules have incorrect indentation (see Layout issues when using Replacements xich/retrie#24)
  • Capture avoidence from binders introduced by RecordWildCards dots
  • Removal of the definition after inlining

Testing

This PR has undergone the following tests:

  • As many distinct golden tests as I could think of. Including previous inlining tests from the hls-retrie-plugin.
  • Running the inline action at a load of locations in the haskell-language-server repo

Fixes #279
Fixes #307
Fixes #1758
Partially fixes #4374 (by rejecting defs with guards, for now)
Fixes #4408 (except for the ghc10 support as noted in the comments)


example.gif


@Aster89 Aster89 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor suggestions or requests of clarification.

Comment thread docs/support/plugin-support.md Resolved
Comment thread haskell-language-server.cabal Resolved
Comment on lines +91 to +93
withCommas [] = []
withCommas [x] = [x]
withCommas (L l e : xs) = L (ensureTrailingComma l) e : withCommas xs

@Aster89 Aster89 Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, you can do this with lenses:

Suggested change
withCommas [] = []
withCommas [x] = [x]
withCommas (L l e : xs) = L (ensureTrailingComma l) e : withCommas xs
withCommas = over _init $ map ensureTrailingComma'
where ensureTrailingComma' (L l e) = L (ensureTrailingComma l) e

I'm still new to them, so I'm just spotting opportunities to practice with it

λ> over _init (map (*10)) ([] :: [Int])
[]
λ> over _init (map (*10)) ([1] :: [Int])
[1]
λ> over _init (map (*10)) ([1..3] :: [Int])
[10,20,3]

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to spend some time to learn how to use lenses. It reads quite clearly.


import ConPatDef (R (R), e)

-- without DisambiguateRecordFields a pattern label resolves by

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a test for when DisambiguateRecordFields is enabled?

Comment thread plugins/hls-retrie-plugin/README.md Resolved
-- the name is imported into the defining module
case map greImportModule (gre_imp gre) of
m : _ -> Just $ Importable m (memberForm gre)
[] -> Nothing -- is this possible?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we think it's not, shall we log it, somehow, so it's easier to tell from a --debug run that this corner case was hit?

I think this is what that logging stuff you were removing might come into play. But I've not used it myself 😆

@fendor fendor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review, will continue tomorrow

Comment thread stack.yaml Resolved
Comment thread stack-lts24.yaml Resolved
Comment on lines +220 to +225
runWithRetrie :: Session a -> IO a
runWithRetrie = runSessionWithTestConfig def
{ testDirLocation = Left testDataDir
, testConfigCaps = codeActionResolveCaps
, testPluginDescriptor = testPlugins
} . const

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick, this does not affect whether we merge the PR as is.

Using Left testDataDir copies all test data into a temporary directory and then runs HLS on all of them. This can make tests trickier to write correctly (i.e., without race conditions) as more files are involved that are stand alone.
Generally, I think it is good practice to only copy the files you actually need for a test. This makes the test faster and also more reliable.
To do this, you need to construct a VirtualFileTree and supply it here as the Right constructor.
Let me know, whether you would like to change this and need more pointers.

Comment thread docs/contributing/plugin-tutorial.md Resolved
Comment thread docs/contributing/plugin-tutorial.md Resolved

@fendor fendor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good to me, some nitpicks, some suggestions. The only thing that needs more dicussion is getCPPmodule, this can be very expensive.

Comment on lines +305 to +308
-- warn and don't make any edits
pluginSendNotification SMethod_WindowShowMessage $
ShowMessageParams MessageType_Warning msg
return (Right ca)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are warnings, why are we still offering the code actions?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a PluginInvalidUserState or a PluginInternalError instead? It is an error in the sense that we tried to do the inline, failed with a message and didn't make any changes. This path is to indicate to the user that if they fix their code they can still retry the action.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user fixed their code, won't the client request new code actions? Or does the editor cache this so aggressively that you can reuse the same code action after a change?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it would be a new action. What did you mean in your original point, I think I’m missing something?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are sending a warning, but we are still producing Right ca, which reads to me as we are returning possible code actions, even though we shouldn't, according to the comment?
Does that mean, we offer code actions, even though we say we couldn't compute something?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it’s just an empty, successful action with no edits. Is there a better way to do this?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning Right []? I don't feel like we should be offering any code actions if we can't compute the text edit, what would happen if the user requests it? Not doing anything?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention in the original plugin was to allow a subset of modules to error out so if, say, one module failed because there was some unsupported syntax it would warn the user that one file was not rewritten but still apply the edits to all of the other files. Within one module it's all-or-nothing so you wouldn't be left with a half edited file. Let me know which behaviour is better, I'm fine either way.

case outcome of
Left msg -> do
-- warn and don't make any edits
pluginSendNotification SMethod_WindowShowMessage $

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be logged, or does our infrastructure log MessageType_Warning by default?

-> CodeAction
-> RunRetrieInlineThisParams
-> ExceptT PluginError (HandlerM Config) CodeAction
resolveInlineThis recorder state ca RunRetrieInlineThisParams{..} = ExceptT $ do

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function feels tricky to read, perhaps it would make sense to split this up into multiple functions with descriptive names?

Usually, I try to structure such functions like this:

  • Should produce something for this call? (E.g., we can offer rewrite code actions)
  • Prepare and validate the work (use custom data types here)
  • Transform the validated work to the actual change
  • Transform the actual change to whatever format we need (in this case, CodeAction)

Perhaps something tlike that would be useful here as well?

editParams =
WorkspaceEdit (Just $ asEditMap $ concat replacements) Nothing Nothing

return (errors, editParams)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to return anything when there are errors?

Comment on lines +878 to +885
getCPPmodule
:: Recorder (WithPriority Log)
-> IdeState
-> HscEnv
-> FixityEnv
-> FilePath
-> IO (CPP AnnotatedModule)
getCPPmodule recorder state session fixities t = do

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a rule of its own?

Probably would require us to also create rule for FixityEnv 🤔

Comment on lines +888 to +905
let parseWith contents = do
modSummary <- msrModSummary <$>
useOrFail state "Retrie.GetModSummary" (CallRetrieInternalError "file not found") GetModSummary nt
let ms' =
modSummary
{ ms_hspp_buf =
Just (stringToStringBuffer contents)
-- keep comments, as GetParsedModuleWithComments does
, ms_hspp_opts =
ms_hspp_opts modSummary
`gopt_set` Opt_KeepRawTokenStream
`gopt_unset` Opt_Haddock
}
logWith recorder Info $ LogParsingModule t
parsed <- evalGhcEnv session (parseModule ms')
`catch` \e -> throwIO (GHCParseError nt (show @SomeException e))
transformA (fixAnns parsed) (fix fixities)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite expensive, when is this executed? I think this should be a rule, and I don't know why we are unsetting Opt_Haddock?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is and initially I thought to exclude this because of the performance implications, but it is also a really nice feature to have (especially for us working on HLS).

The getCPPmodule function is called for every target file (so for inlining/unfolding, every file that contains a call to the function being inlined).

On line 914 we have the following CPP check:

  if LangExt.Cpp `EnumSet.member` extensions
    then parseCPP parseWith contents
    else NoCPP <$> transformA (fixAnns pm) (fix fixities)

If CPP is not used then we enter the fast path and re-use the parse from the GetParsedModuleWithComments rule. If CPP is enabled, there is a somewhat questionable algorithm that forks the file into every possible CPP branch and parses all of them (see https://github.com/xich/retrie/blob/b118ec69f172f25ae3dd15baeb18801259aad69f/Retrie/CPP.hs#L179) - that's what the parseWith is for. This allows us to apply rewrites across all of the CPP branches, which is really nice, but it can be horribly slow if there are too many branches (on that point there is also an improvement to retrie that would drastically reduce the number of forks, but I haven't had time to look into this).

As for unsetting haddock, it is copied from getParsedModuleWithCommentsRule (see

-- For that use case you might want to use `getParsedModuleWithCommentsRule`
and
getParsedModuleWithCommentsRule :: Recorder (WithPriority Log) -> Rules ()
). I haven't actually checked whether this causes any problems but there are some slight differences in how the comments are represented and given that this is what GetParsedModuleWithComments does it should probably do the same.

@simonhorlick

Copy link
Copy Markdown
Collaborator Author

@fendor Is there a specific recommendation regarding the ghc version branching? I ask because in retrie the style is to use #if __GLASGOW_HASKELL__ < 914 (always with a less-than). I find it easy to read, but I don't know if it's common here or whether there are any drawbacks.

Co-authored-by: fendor <fendor@users.noreply.github.com>
@fendor

fendor commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

@simonhorlick We use MIN_VERSION_ghc pragmas.

We have this guideline in HLS for imports:

{- Note [Guidelines For Using CPP In GHCIDE Import Statements]
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  GHCIDE's interface with GHC is extensive, and unfortunately, because we have
  to work with multiple versions of GHC, we have several files that need to use
  a lot of CPP. In order to simplify the CPP in the import section of every file
  we have a few specific guidelines for using CPP in these sections.

  - We don't want to nest CPP clauses, nor do we want to use else clauses. Both
  nesting and else clauses end up drastically complicating the code, and require
  significant mental stack to unwind.

  - CPP clauses should be placed at the end of the imports section. The clauses
  should be ordered by the GHC version they target from earlier to later versions,
  with negative if clauses coming before positive if clauses of the same
  version. (If you think about which GHC version a clause activates for this
  should make sense `!MIN_VERSION_GHC(9,0,0)` refers to 8.10 and lower which is
  an earlier version than `MIN_VERSION_GHC(9,0,0)` which refers to versions 9.0
  and later). In addition there should be a space before and after each CPP
  clause.

  - In if clauses that use `&&` and depend on more than one statement, the
  positive statement should come before the negative statement. In addition the
  clause should come after the single positive clause for that GHC version.

  - There shouldn't be multiple identical CPP statements. The use of odd or even
  GHC numbers is identical, with the only preference being to use what is
  already there. (i.e. (`MIN_VERSION_GHC(9,2,0)` and `MIN_VERSION_GHC(9,1,0)`
  are functionally equivalent)
-}

Since we usually support many GHc versions, the goal is to write code from the perspective of the latest GHC version, s.t. when we drop an older GHC version, we simply delete the code we don't care about any more and don't need to refactor anything to make use of the latest API improvements.

I would generally recommend the same guidelines for non import statements.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants