semantic/docs/💡ProTip!.md

💡ProTip!

Performance

• You can conveniently build and run semantic with profiling enabled using script/profile. Any arguments you pass to the script will be passed along to semantic. It will archive the results in the profiles folder, and open the run when it’s finished.
• Similarly, you can generate threadscope profiles with script/threadscope.
• To profile you will need to make sure profiteur is installed: stack install profiteur along with ps2pdf: brew install ghostscript.
• The Haskell wiki has some handy rules of thumb for strict evaluation. This can be useful when dealing with performance issues caused by long chains of lazy computations, and when ensuring sequencing (e.g. for parallelism or correct handling of FFI allocations).

Memory Leaks

Space leaks are sometimes introduced into Semantic when holding on to a reference for too long, building up unevaluated expressions, or by keeping around unneeded references. While GHC has a garbage collector that can prevent dangling pointers, it's still possible to create space leaks in connection with lazy evaluation. Space leaks not only consume more memory, but also slow down the garbage collector considerably!

Space leaks can be detected by running semantic with a restricted heap size. Neil Mitchell's blog post describes a method for detecting space leaks by analyzing the stack traces of stack overflows when compiling with a restricted heap size.

Building

stack build --fast is a nice way to speed up local development (builds without optimizations).

Testing

stack build --fast semantic:test builds and runs the unit tests.

• Find out what all the possible test arguments are with stack test --help.

• Focus in on a particular test or set of tests with -m/--match:

    stack test --test-arguments="-m ruby"
  

• Use --skip to run everything but matching tests:

    stack test --test-arguments="--skip ruby"
  

• It can take a while to run them over the whole project. Focus in on a particular module with --test-arguments:

    stack test --test-arguments=src/Data/Range.hs
  

Difftool

git can be configured to open diffs in semantic using git-difftool:

1. Install semantic to the local bin path: stack install :semantic

2. Configure semantic as a difftool:

    git config difftool.semantic.cmd 'semantic diff --patch "$LOCAL" "$REMOTE"'
   

3. Optionally, configure semantic as the default difftool:

    git config diff.tool semantic
   

4. Perform git diffs using semantic by invoking git-difftool:

    # if configured as default
    git difftool
    # otherwise
    git difftool -t semantic
   

5. Bonus round! Optionally, configure git-difftool to never prompt:

    git config difftool.prompt false
   

Editing

• 1. Install ghc-mod from the semantic directory by running:

     stack install ghc-mod

• 2. You'll need the ide-haskell plugins for atom. You can install through apm:

       apm install haskell-ghc-mod ide-haskell ide-haskell-cabal linter linter-ui-default

Ctags Support

You can enable ctags support for the project by installing codex to generate a ctags file. This is often useful because haskell-ide-engine's jump-to-definition feature can break during editing.

    `stack install hasktags`
    `git clone https://github.com/aloiscochard/codex && cd codex && stack install codex`

To generate or update the tags file:

    `codex update`

Symbol support is handled by the symbols-view package in atom or your friendly editor's ctags support. In atom you can use cmd-alt-down and cmd-alt-up to jump to a definition and jump back to your original position. If you're using a vim plugin, they're also bound to ctrl-] and ctrl-t. cmd-shift-r lists all tags in the project.

The symbols-view package only recognizes files named tags in the root of your project so you'll have to add a symlink:

    `ln -s codex.tags tags`

Alternatively, you can replace symbols-view with joshvera/tags-view in your atom packages which adds support for jumping to absolute paths (if you want to navigate to a haskell dependency or base library in stackage) and recognizes codex.tags files.

    `git clone https://github.com/joshvera/tags-view ~/.atom/packages/tags-view`

Then disable the symbols-view package.

Semantic documentation in Dash

You can generate a semantic docset and import it into Dash locally. To do so run the script/haddock first to ensure Haddock documentation is generated. Then run script/docset. This should generate .docset/semantic.docset in the semantic repo. The last step is to import the semantic.docset into Dash. Open dash, open preferences, select the 'Docsets' tab, click the + icon to add a new docset, and direct the file browser to semantic/.docsets/semantic.docset.

Working with grammar datatypes

haskell-tree-sitter includes some TemplateHaskell machinery to generate a datatype from a tree-sitter parser’s symbol table. You can generally guess the constructors of that type by turning the snake_case production names from the tree-sitter grammar into UpperCamelCase names, but you can also have the compiler dump the datatype out in full in the repl:

    λ :info Language.JSON.Grammar.Grammar

Voilà! You’re now looking at the source code for the datatype generated from the symbol table:

data Language.JSON.Grammar.Grammar
  = Language.JSON.Grammar.END
  | Language.JSON.Grammar.AnonLBrace
  | Language.JSON.Grammar.AnonComma
  | Language.JSON.Grammar.AnonRBrace
  | Language.JSON.Grammar.AnonColon
  | Language.JSON.Grammar.AnonLBracket
  | Language.JSON.Grammar.AnonRBracket
  | Language.JSON.Grammar.String
  | Language.JSON.Grammar.Number
  | Language.JSON.Grammar.True
  | Language.JSON.Grammar.False
  | Language.JSON.Grammar.Null
  | Language.JSON.Grammar.HiddenValue
  | Language.JSON.Grammar.Object
  | Language.JSON.Grammar.Pair
  | Language.JSON.Grammar.Array
  | Language.JSON.Grammar.AuxObjectRepeat1
  | Language.JSON.Grammar.AuxArrayRepeat1
  | Language.JSON.Grammar.ParseError
  	-- Defined at src/Language/JSON/Grammar.hs:10:1
instance Bounded Language.JSON.Grammar.Grammar
  -- Defined at src/Language/JSON/Grammar.hs:10:1
instance Ord Language.JSON.Grammar.Grammar
  -- Defined at src/Language/JSON/Grammar.hs:10:1
instance Eq Language.JSON.Grammar.Grammar
  -- Defined at src/Language/JSON/Grammar.hs:10:1
instance Enum Language.JSON.Grammar.Grammar
  -- Defined at src/Language/JSON/Grammar.hs:10:1
instance Show Language.JSON.Grammar.Grammar
  -- Defined at src/Language/JSON/Grammar.hs:10:1

GHCi

The Haskell interactive repl (GHCi) allows you to quickly typecheck your work and test out ideas interactively. It’s always worth having a repl open, but we’ve particularly tuned some workflows, e.g. semantic assignment development, for the repl.

Configuration

We configure ghci with defaults & macros for use with semantic via the .ghci file at the project root, and you can further customize its behaviour via the ~/.ghci file.

Full docs for ghci can be found in the user’s guide.

Managing history

ghci uses the haskeline package to perform the readline-like navigation, search, and management of history, as well as tab-completions. haskeline has several user preferences and custom key bindings which can be configured per-user via the ~/.haskeline file. For example, these preferences cause haskeline to include only one instance of repeated commands in history, and to place no upper bound on the number of commands it will remember:

historyDuplicates: IgnoreAll
maxHistorySize: Nothing

Pretty-printing

By default, GHCi prints the results of expressions using their Show instances, which can be particularly difficult to read for large recursive structures like Terms and Diffs. The project’s .ghci file provides :pretty & :no-pretty macros which respectively enable & disable colourized, pretty-printed formatting of result values instead. These macros depend on the the pretty-show & hscolour packages.

Since :reloading resets local bindings, the .ghci file also provides a convenient :r macro which reloads and then immediately re-enables :pretty.

You can use :pretty & :no-pretty like so:

λ :no-pretty
λ Data.Range.Range <$> [1..3] <*> [4..6]
[Range {start = 1, end = 4},Range {start = 1, end = 5},Range {start = 1, end = 6},Range {start = 2, end = 4},Range {start = 2, end = 5},Range {start = 2, end = 6},Range {start = 3, end = 4},Range {start = 3, end = 5},Range {start = 3, end = 6}]
λ :pretty
λ Data.Range.Range <$> [1..3] <*> [4..6]
[ Range { start = 1 , end = 4 }
, Range { start = 1 , end = 5 }
, Range { start = 1 , end = 6 }
, Range { start = 2 , end = 4 }
, Range { start = 2 , end = 5 }
, Range { start = 2 , end = 6 }
, Range { start = 3 , end = 4 }
, Range { start = 3 , end = 5 }
, Range { start = 3 , end = 6 }
]

Working in Assignment

When working in assignment, some setup is required. This macro automates that by automatically importing the necessary modules and outputs an example command. If you provide the language you are working with as an optional parameter, the example command is formatted for that language's specific needs (parser, example file extension, etc.).

The macro is defined as:

:{
assignmentExample lang = case lang of
  "Python" -> mk "py" "python"
  "Go" -> mk "go" "go"
  "Ruby" -> mk "rb" "ruby"
  "JavaScript" -> mk "js" "typescript"
  "TypeScript" -> mk "ts" "typescript"
  "Haskell" -> mk "hs" "haskell"
  "Markdown" -> mk "md" "markdown"
  "JSON" -> mk "json" "json"
  _ -> mk "" ""
  where mk fileExtension parser = putStrLn ("example: fmap (() <$) . runTask . parse " ++ parser ++ "Parser =<< Semantic.Util.blob \"example." ++ fileExtension ++ "\"") >> return ("import Parsing.Parser\nimport Semantic.Task\nimport Semantic.Util")
:}

:def assignment assignmentExample

And is invoked in GHCi like:

λ :assignment Python

The output produces a one line expression assuming the syntax to assign is in a file named example with the relevant programming language extension:

quieterm <$> parseFile pythonParser "example.py"

Inspecting TreeSitter ASTs

Inspecting the parse tree from TreeSitter can be helpful for debugging. In GHCi, the command below allows viewing the TreeSitter production name of each node in the TreeSitter AST:

import TreeSitter.Java
fmap nodeSymbol <$> parseFile javaASTParser "example.java"

Using Threadscope

Threadscope is a tool for profiling the multi-threaded performance of Haskell programs. It allows us to see how work is shared across processors and identify performance issues related to garbage collection or bottlenecks in our processes.

To install threadscope:

1. Download a prebuilt binary from https://github.com/haskell/ThreadScope/releases .
2. chmod a+x the result of extracting the release.
3. brew install gtk+ gtk-mac-integration.
4. profit.