Yarith · December 8, 2018 21:26 · Dec 8, 2018 · Dec 8, 2018 · Dec 8, 2018 · Dec 8, 2018
diff --git a/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md b/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md
@@ -35,4 +35,4 @@ I tried hours after hours to change everything around `ghc-mod` to get rid of th
 
 ## Optional: Install the ghc-mod cli to access some data without the `Haskell IDE Engine`
 If you want to compile the matching `ghc-mod-5.9.0` for `ghc-8.2.2` i have already a file about it:
-[Notes about compiling ghc-mod 5.9.0 with stack](#file-compile-ghc-mod-5.9.0-with-stack-md)
+[Notes about compiling ghc-mod 5.9.0 with stack](#file-compile-ghc-mod-5-9-0-with-stack-md)
diff --git a/elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md b/elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md
@@ -1,6 +1,6 @@
 # elm compiler - what is preventing us to inspect the code with ghc-mod?
 Should you need help for setting up the `elm compiler` directory or the `Haskell IDE Engine` i have another file that discribes the installation:
-[Configure `elm compiler` for use with `Haskell IDE Engine`](#file-prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md)
+[Configure `elm compiler` for use with `Haskell IDE Engine`](#file-prepare-elm-compiler-with-haskell-ide-engine-for-vscode-md)
 
 This document explains and shows a way to fight the error by modify some `.hs` files:
 ```

diff --git a/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md b/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md
@@ -31,8 +31,8 @@ EXCEPTION: browse:
     Workaround: use -fobject-code, or compile this module to .o separately.
 ```
 I tried hours after hours to change everything around `ghc-mod` to get rid of the error. Nothing seemed to help. Today i have looked which module is causing the errors and finally was able to open the `elm compiler` project with full type information in every module. I have took a slightly detailed document with notes about my process and saved it in an own file:
-[elm compiler - what is preventing us to inspect the code with ghc-mod?](#file-elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md)
+[elm compiler - what is preventing us to inspect the code with ghc-mod?](#file-elm-compiler-ghc-mod-can-not-provide-info-for-all-modules-md)
 
 ## Optional: Install the ghc-mod cli to access some data without the `Haskell IDE Engine`
 If you want to compile the matching `ghc-mod-5.9.0` for `ghc-8.2.2` i have already a file about it:
-[Notes about compiling ghc-mod 5.9.0 with stack](#file-compile-ghc-mod-5.9.0-with-stack.md)
+[Notes about compiling ghc-mod 5.9.0 with stack](#file-compile-ghc-mod-5.9.0-with-stack-md)
diff --git a/compile-ghc-mod-5.9.0-with-stack.md b/compile-ghc-mod-5.9.0-with-stack.md
@@ -2,7 +2,8 @@
 Notes about the compilation of ghc-mod 5.9.0 for ghc 8.2.2.
 
 ## Working yaml file for the master branch at changeset 96c2207
-Used commit: [changeset 96c2207](https://github.com/DanielG/ghc-mod/commit/96c2207f0e9019f958d39582d4bf4af3bc9469d2)
+Used commit: [changeset 96c2207](https://github.com/DanielG/ghc-mod/tree/96c2207f0e9019f958d39582d4bf4af3bc9469d2)
+Maybe the newer commit with [changeset e5b7daf](https://github.com/alanz/ghc-mod/tree/e5b7daf1a2c949b2d2900ae11b13a267ed25eedb) which is used by the ```Haskell IDE Engine``` in my today installation maybe work too. Have not tested it standalone.
 
 Working stack.yaml file:
 ```yaml

diff --git a/elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md b/elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md
@@ -0,0 +1,142 @@
+# elm compiler - what is preventing us to inspect the code with ghc-mod?
+Should you need help for setting up the `elm compiler` directory or the `Haskell IDE Engine` i have another file that discribes the installation:
+[Configure `elm compiler` for use with `Haskell IDE Engine`](#file-prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md)
+
+This document explains and shows a way to fight the error by modify some `.hs` files:
+```
+EXCEPTION: browse:
+    Error: bytecode compiler can't handle unboxed tuples and sums.
+    Possibly due to foreign import/export decls in source.
+    Workaround: use -fobject-code, or compile this module to .o separately.
+```
+
+If you checkout the `Elm Compiler` and open it with `Visual Studio Code` with configured `Haskell IDE Engine` you will notice really fast, that for some modules are no information available. This document tracks my journey to answer: Why it does not work and what we can change? For me it is enough that we can at least inspect our modules again. This solution changes the code in this way that the `ghc-mod` never enters the `Interpreter Mode` which gets activated if there are some language extensions used.
+
+## Find the module which prevents inspecting all modules
+There are some files in the `elm/compiler` repository that can not be inspected with `ghc-mod` because of the `unpacked tuple` error. To find out which module is causing it you can use this `bash command`:
+```bash
+ghc-mod debug | while read f
+do
+    regex="([a-zA-Z0-9.]+) \(.*\.hs\)"
+    if [[ $f =~ $regex ]]
+    then
+        module="${BASH_REMATCH[1]}"
+        echo "*** Module: $module ***"
+        ghc-mod browse $module
+    fi
+done > project-modules-symbols.txt 2>&1
+```
+This will try to get all symbols for every module that is declared in the project. After it is done you can look for errors. I know that there will be done a second run, but i dont know how i can add a distinct for the module names.
+
+### Tracing back the source
+The `Bump` module is the first module that could not be loaded in our list. Lets start the investigation on this module.
+1. Lets look for the `Bump`. It depends on the other 4 modules. We look up if they could be loaded and found out that the modules `Elm.Bump` and `Elm.Project` also could not be loaded. 
+2. Lets look deeper in the `Elm.Bump` module. It depends on 12 other modules. Here is `Elm.Project` also used, which we already know that it could not be loaded. Every other module could be loaded.
+3. Lets look deeper in the `Elm.Project` module. It depends on 15 other modules. Here only the module `Generate.Output` could not be loaded.
+4. Lets look deeper in the `Generate.Output` module. It depends on 22 other modules. All of them could be loaded.
+5. Lets look if all modules which could not be loaded depends on `Generate.Output`. After checking all modules manually, we find out that every other depends on the module `Generate.Output` or a module which depends on it.
+
+It seems so that the `Generate.Output` has something which ghc-mod does not like with our current configuration. 
+
+## Lets inspect why our found source can not be loaded
+We found out that the module `Generate.Output` itself can not be loaded by `ghc-mod`. The tests are done with `ghc-mod -v -g "-O0" -g "-fobject-code" browse` followed by the module name.
+
+Lets delete some code from the module until the error disappears.
+1. We comment everything out and leave only the the type `Mode` and the export of the type. This should always work. After a test with `ghc-mod browse` the error is gone.
+2. Comment all imports in. The error appeared again. So there must be a dependency which is causing this.
+3. Look for the imports that is causing the error. The import of one module `Generate.Functions` alone is enough to cause the switch to the `Interpreter Mode`, but we get still results without an error. The module `Generate.Html` is causing the `Interpreter Mode` too without an error. All other modules are not causing the `Interpreter Mode` and if they are with the other two, there happens no error. 
+4. Which module is not able to run within `Interpreter Mode`? Tested by letting `Generate.Functions` imported and trying out the other imports. So the result is these modules are not working with `Interpreter Mode`: `Elm.Compiler`, `Elm.Compiler.Module`, `Elm.Compiler.Objects`, `Elm.Project.Json`, `Elm.Project.Summary`, `File.Args`, `File.Crawl`, `File.IO`, `Reporting.Exit`, `Reporting.Exit.Make`, `Reporting.Task`, `Stuff.Paths` and `Terminal.Args`. These are quite many.
+
+So we found out: We have 2 modules that are causing the `Interpreter Mode` and 13 modules that are not working with the `Interpreter Mode`. So what is the `Interpreter Mode`? Why is it neccessary and why is it causing problems? 
+So we know from the info output of `ghc-mod`, that it is normal that it is enabled when the language extensions `TemplateHaskell`, `QuasiQuotes`, `PatternSynonyms` are used. `QuasiQuotes` for example is used for the modules `Generate.Functions`, `Generate.Html`, `Develop.Generate.Help`. `QuasiQuotes` is used to extend the syntax of haskell programs, here it adds a simpler form for multiline raw strings and using of quotes without the need to escape them. `TemplateHaskell` is used only for the `runIO` function.
+
+## Get rid of the language extensions that changes the behavior of ghc-mod
+### Translate the `QuasiQuote` multiline strings to regular multiline strings
+Haskell has already multiline strings, but they require for every line at the end the `new line` sequence and the escape of the `whitespace` characters. Also we need to escape every `backslash` and `double quote`. Because in the code there are only a few places i go manually through the code and replace it. At this stage it is only a test if it is enough to solve our problem with `ghc-mod`.
+
+Steps for the modification:
+1. Remove the `QuasiQuote` language extensions
+2. Remove the `Text.RawString.QQ` module from the imports
+3. Remove the surrounding `[r||]` from a multiline text and use double quotes instead. 
+4. Escape all `double quote`s and `backslash`es in the content.
+5. Add after every line `\n\` except the last and before every new line `\`. For empty lines add `\\n\`.
+6. Repeat starting at 3. until no `[r||]` block are left.
+
+After the modification i run the `ghc-mod browse Generate.Functions` command. And it runs through without error! Lets rerun the bash script above for the generation of the file `project-modules-symbols.txt`. After it is done we can see how many modules are now inspectable with `ghc-mod`.
+
+There are only 3 modules which can not be loaded by `ghc-mod`. These are: `Main`, `Develop`, `Develop.StaticFiles`. In this order they are depend on the next one. While only `Develop.StaticFiles` contains the last remaining language extension which causes the switch to the `Interpreter Mode` in `ghc-mod`.
+
+## Remove the need of `TemplateHaskell` in `Develop.StaticFiles`
+At this point we already know that the `runIO` from the module `Language.Haskell.TH` requires the language extension `TemplateHaskell`. Now we need to figure out how we can change the code so the `runIO` function is not required anymore. At this point i have no clou if and how it is possible.
+
+Next step is reading the docs, maybe there is a standard equivalent already documented like it is for the do notation. So, after reading the docs it is clear, that the execution of the part between `$(` and `)` is executed on compile time and constructs an expression which contains the resulting data. In `Develop.StaticFiles` they are all `ByteString`s. The function `bsToExp` constructs a `Q Exp` from an input of type `ByteString` which must be converted to the build time constant `ByteString` with the `$()` operation.
+
+We need an alternative approach for the module `Data.FileEmbed`. These are basically resources which i know from other languages. It seems so the compiler has no standard syntax to embed a file as a `ByteString` on compile time. I think for this problem is no alternative solution available. So i will declare an `embed` with the `IO ByteString` result of the loader functions. In my use case i will only analyze the application but never compile it directly for execution.
+
+### Changing the file
+First remove the `TemplateHaskell` macro line and then remove the imports of the modules `Data.FileEmbed` and `Language.Haskell.TH`.
+
+I added following function to the `Develop.StaticFiles` module:
+```haskell
+-- TEMPLATE HASKELL ALWAYS EMPTY MOCK
+embed :: IO BS.ByteString -> BS.ByteString
+embed _ =
+  ""
+```
+
+And replaced all usages of the template syntax `$()` with the newly created `embed` function. The calls to the `Develop.StaticFiles.Build` are still there so we can find it with `Find all references` from the IDE. 
+```haskell
+-- ELM
+
+
+elm :: BS.ByteString
+elm =
+  embed(Build.compile)
+
+
+
+-- CSS
+
+
+css :: BS.ByteString
+css =
+  embed(Build.readAsset "styles.css")
+
+
+
+-- FONTS
+
+
+codeFont :: BS.ByteString
+codeFont =
+  embed(Build.readAsset "source-code-pro.ttf")
+
+
+sansFont :: BS.ByteString
+sansFont =
+  embed(Build.readAsset "source-sans-pro.ttf")
+
+
+
+-- IMAGES
+
+
+favicon :: BS.ByteString
+favicon =
+  embed(Build.readAsset "favicon.ico")
+
+
+waiting :: BS.ByteString
+waiting =
+  embed(Build.readAsset "waiting.gif")
+```
+
+This change has basically removed the resources for the `elm reactor` and `index.html` creation. If you do not need both you could even use the compiled executable.
+
+Lets test if we can now get the module symbols with the `ghc-mod browse Develop.StaticFiles` command. It worked! Test if `Develop` and `Main` are working too. Alright, everything worked! 
+
+### Convert the files to `ByteString` literal so the application works as it should
+You can do this step if you need to start the `Elm Compiler` with the reactor. But for my case it is not neccessary so i will stop at this point my journey and continue to inspect the source with full HIE support. 
+
+
+🎉 🎉 🎉
diff --git a/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md b/prepare-elm-compiler-with-haskell-ide-engine-for-vscode.md
@@ -0,0 +1,38 @@
+# Configure `elm compiler` for use with `Haskell IDE Engine`
+This document describes the steps to get a working copy of the `elm compiler` source code which can be inspected via the Haskell IDE Engine. 
+
+My setup used the `ghc-8.2.2`. The latest lts-resolver for this version was: `lts-11.22`. This `stack` resolver is used for all compilations.
+
+## Install the `Haskell IDE Engine 4.0.1` with `stack` for `ghc-8.2.2`
+1. Clone the repository
+   `git clone https://github.com/haskell/haskell-ide-engine.git -b 0.4.0.1 --recursive`
+   This gets the version with the tag `0.4.0.1` on the commit [df6da63](https://github.com/haskell/haskell-ide-engine/tree/df6da6308fb5de672ad82ed5cb4dfeb91050d052).
+2. Install the `Haskell IDE Engine` via `stack install --stack-yaml stack-8.2.2.yaml --resolver lts-11.22`.
+3. Prepare the hoogle database via `stack --stack-yaml stack-8.2.2.yaml --resolver lts-11.22 exec hoogle generate`
+   If you are not doing this step you will probably get following warning later in `vscode`: ```No hoogle db found. Check the README for instructions to generate one```
+
+## Install and configure the `Haskell Language Server` extension
+Install and Configure the `vscode` extension `Haskell Language Server` to use the `~/.local/bin` directory if you have not added it to the `$PATH` environment variable.
+
+## Setup the elm directory
+1. Clone the repository
+   `git clone https://github.com/elm/compiler.git`.   
+   At the time of writing the current commit was [a936e95](https://github.com/elm/compiler/tree/a936e95a4d915a41f501ce70878675e20df03d60).
+2. Create a `stack.yaml` file with `stack init` and then open the `stack.yaml` file with an editor and change the resolver to `lts-11.22`.
+3. Execute `stack build`
+4. Maybe it requires you to add some `extra-deps` just add them as suggested and run `stack build` again. At some point i removed them again and it does still work.
+
+## Help! Not for all modules are information available
+Maybe you will encounter soon the error:
+```
+EXCEPTION: browse:
+    Error: bytecode compiler can't handle unboxed tuples and sums.
+    Possibly due to foreign import/export decls in source.
+    Workaround: use -fobject-code, or compile this module to .o separately.
+```
+I tried hours after hours to change everything around `ghc-mod` to get rid of the error. Nothing seemed to help. Today i have looked which module is causing the errors and finally was able to open the `elm compiler` project with full type information in every module. I have took a slightly detailed document with notes about my process and saved it in an own file:
+[elm compiler - what is preventing us to inspect the code with ghc-mod?](#file-elm-compiler-ghc-mod-can-not-provide-info-for-all-modules.md)
+
+## Optional: Install the ghc-mod cli to access some data without the `Haskell IDE Engine`
+If you want to compile the matching `ghc-mod-5.9.0` for `ghc-8.2.2` i have already a file about it:
+[Notes about compiling ghc-mod 5.9.0 with stack](#file-compile-ghc-mod-5.9.0-with-stack.md)
diff --git a/compile-ghc-mod-2.0.1-with-stack.md → compile-ghc-mod-5.9.0-with-stack.md b/compile-ghc-mod-2.0.1-with-stack.md → compile-ghc-mod-5.9.0-with-stack.md
@@ -1,5 +1,5 @@
 # Compile ghc-mod on OpenSUSE Tumbleweed (2018-12-04)
-Notes about the compilation of ghc-mod 2.0.1 for ghc 8.2.2.
+Notes about the compilation of ghc-mod 5.9.0 for ghc 8.2.2.
 
 ## Working yaml file for the master branch at changeset 96c2207
 Used commit: [changeset 96c2207](https://github.com/DanielG/ghc-mod/commit/96c2207f0e9019f958d39582d4bf4af3bc9469d2)

diff --git a/compile-ghc-mod-2.0.1-with-stack.md b/compile-ghc-mod-2.0.1-with-stack.md
@@ -0,0 +1,43 @@
+# Compile ghc-mod on OpenSUSE Tumbleweed (2018-12-04)
+Notes about the compilation of ghc-mod 2.0.1 for ghc 8.2.2.
+
+## Working yaml file for the master branch at changeset 96c2207
+Used commit: [changeset 96c2207](https://github.com/DanielG/ghc-mod/commit/96c2207f0e9019f958d39582d4bf4af3bc9469d2)
+
+Working stack.yaml file:
+```yaml
+resolver: lts-11.22
+packages:
+- location: .
+- location: ./core
+extra-deps:
+- monad-journal-0.7.2
+- either-4.4.1.1
+- free-4.12.4
+- hlint-2.0.8
+- haskell-src-exts-1.19.1
+- extra-1.5.3
+- cabal-helper-0.8.1.2@sha256:da0698b950fd28fef469c1092cda5f0e71ac56204961afd9d800fff2cc908607
+- cabal-plan-0.4.0.0@sha256:0bb2b5194d7f757437a55b98d8c35312d362f5912268668dc2faa2ca977b3b98
+- pretty-show-1.8.2@sha256:e16e0174bf4ad3dae49fea69b1f65ca9d8d4b61b3d80c78fbe3320f2181e92e9
+flags: {}
+extra-package-dbs: []
+```
+After saving the `stack.yaml` just execute `stack install`.
+
+## Troubleshooting
+### How to solve `ld -lgmp` error?
+Install package `gmp-devel`
+### How to solve `ld -ltinfo` error?
+Install package `ncurses-devel`
+### zlib package does not compile
+Install package `zlib-devel`
+```
+Configuring zlib-0.6.2...
+Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: Missing dependency on a foreign
+library:
+* Missing (or bad) header file: zlib.h
+* Missing C library: z
+```
+### Multi-way if-expressions need MultiWayIf turned on
+Add to the ghc-mod-core.cabal under the section Default-Extensions the `MultiWayIf` entry.
No results found