In moving from v. 0.15 to 0.16, haskell-lsp changed the type of the
value which ended up being passed to `pretty` from `Maybe Text` to
`Maybe LSP.NumberOrString`, thereby breaking the line of code which is
removed in this commit.
After discussion with @ndmitchell, it was observed that this code was
never useful, and fixing it was not worth the trouble.
* Put information about the compiler pass in the _source field of Diagnostic
It's useful when looking at the hie-core command-line output to see where
a problem is coming from.
* include test code in hie-core's own cradle
* #2326, GHC 8.4 compatibility
* Fix up CI
* Add a Shake lower bound
* Upgrade to a hie-bios which is GHC 8.4 compatible
* Add a GHC 8.4 stack
* Fix HLint again
* language: check dflags for errors
We add a check when we build the dflags for cases that will lead to a
failed build and emmit a clearer error message. Currently this only
includes a check, to see whether the current installed unit id is also
imported as a package from the package database.
* exclude ghc-prim from check
* exclude code generation from dflag check
* add an internal option to turn dflags check off
* Adding more info to dor
* Adding fields to the graph
* Spelling mistakes and params
* Removing single line function
* Moving the duplicated code
* Revert "Moving the duplicated code"
This reverts commit 520b5d9b0d.
* Adding more structure to graph
* Working tests with graph which is more easy to understand and change.
have few more tests to migrate
* Adding edge details
* One more to go
* Adding more verbose field information
* All tests now tests the fields that are being added
* Removing unused Import
* defined not refined
* Unused imports
* Names need to be consitent
* Acutal and Expected, not expected and expected
* Removing unused export
* Lints
It has been deprecated for a while now with a hint to use `filterWithKey`
instead. We should add
```haskell
filter : (v -> Bool) -> TextMap v -> TextMap v
```
after the next SDK release instead. This is tracked in #2760.
At the top level we don't care if bindings are recursive or not. The logic
that takes care of this ignorance is currently mixed with the actual logic
for converting the bindings.
This PR separates these concerns properly. This is also in preparation for
another upcoming change which needs access to all top level bindings.
* Add missing alpha-equivalence case.
* Type-level nat is not serializable.
* Convert decimal primitives as numeric primitives if feature is available.
* Convert decimal literals to numeric if available.
* Better approach to decimal primitves.
* Fixing BEDecimalFromText
* Add issue number
* Set Numeric feature at v1.7
This allows us to run "damlc build" in Bazel rules which is a
prerequisite for doing anything with DAML packages as they don’t work
with "damlc package".
This make the build of upgrade modules a bit more quiet. The warnings
appear because in the generated generic instances there is an
unrecognised {-# NO_OVERLAPP #-} pragma annotation.
* add Numeric.java
* ledger-api: rename `decimal` field to `numeric` in value protobuf
* Address Gerolf's comment
* ledger-api: add missing renammings
* ledger-api: relax syntax of numbers that can be sent as numerics
* extractor: fix
* leger-api: change format of number though ledger api
* daml-lf: fix numeric regexp
* ledger: fix tests
* Templates and scenario for generic transfers
* Simplify transfer workflow by self-issuing cash
* Generic swap in the same style as Transferrable
* Use Template class constraints instead of *Instance constraints
* Shorter asset name for transferrable/swappable templates
* Shorten some stuff
* Simplify swap acceptance, remove dependency on transfers, add some checks in scenarios
* Remove FlexibleContexts extension which should be on by default
* Move Asset typeclass and Cash instance to separate module
* Rename Swap module to Swappable
* Revert change to VERSION file
* language: compile everything in the source directory
This removes the need to specify a 'main'. Instead we 'source' in
daml.yaml should point to the source root directory.
Currently, recursive top level functions whose type is boundedly polymorphic
require a type annotation to make it through the conversion to DAML-LF.
This PR changes the conversion slightly such that a type annotation is n o
longer required.
This fixes#2669.
Currently, we use `Safe 1` as the safety of the `BEMapEmpty` primitive. This
is clearly wrong since `BEMapEmtpy` is not supposed to be applied to a value
(but only to a type). Thus, it must be `Safe 0`. This has not caused any
problems in the past because the type checker would have caught any
application of `BEMapEmpty` to a value.
Noticed this while trying to debug the segfaults.
I don’t have a concrete case where this causes issues (usually we only
call this once on startup so leaks are not an issue) but we might as
well do it properly.
Both language extension are needed to improve the UX of generic templates.
Without them templates a la
```
template Template (A t) => B t with
```
would produce an error that we need `FlexibleContexts` for the
`Template (A t)` context and a warning suggesting to simplify it to
`AInstance t`. The latter would expose an implementation detail we prefer
to hide.
* Add test case for default method signature
* Fix default method docs
* lint
* Refactor ty calculation in getMethodDocs
* Add test for multiple names in one method type sig
* daml visualize with lsp
* Working command execution with ide
* Have to call visual now
* Split function
* Wrong constuctions of world or the module
* Working almost
* Have to gather all modules
* Now works end to end. Considers all the daml files in workspace
* removing unused code
* removing unused dependency
* Removing unused JS code and better error message
* Function re-arrange
* Removing us of heaad, instead handling error
* Format and rearranging
* Removing unused assignment
* Formating everything
* missed dependency
* Generating visual only for the file that is open
* Using modules from generated pkg, error messages
* Bringing back the commands
* Names and formating
* consitent error message
* Removing Just pattern match using _use now
* package
* Moving visual to a different module
* Function simple
* Removing Rule as it increases memeory consumption
* TODO LSP error
* White space
* Fmt things
* Handling if command was executed for a non daml files
* Better error message
* debugging
* Fix executeCommand
* Adding tests to execute command. Thanks moritz
* Unwanted lines
* Spaces and comment
* 3rd time ?
* Spaces are white
* assert equal instead of assert bool
* language: use generic templates for upgrades
Instead of generating templates we now have generic upgrade/rollback
templates in the standart library and we just create instances of in the
migration project. I will update the documentation in a follow up PR.
* Delete obsolete proto3-suite patch
My patch has been upstreamed so no need to keep it around in our repo.
* Upgrade proto3-wire
* Adapt to changes in proto3-suite
* language: smoothing out the migration experience
Some improvements to make the migration process simpler when migrating
between packages with the same name, e.g. foo-1.0.0 -> foo-2.0.0. This
is the main use case.
This switches the creation of the archive in `daml build` from
`zip-archive` to `zip`. This has a few advantages:
1. It gets rid of lazy IO for reading all the interface and source
files. This avoids the high usage of file handles in `daml build`.
2. It seems to be a slight improvement in max memory usage and runtime
and a giant improvement in allocations (but I think the latter
probably comes primarily from the fact that the locations are moved to
the bzip C library). The improvement in max memory usage is less than
I expected so probably there is still something off somewhere.
For now, I only switched over `createArchive`. Archive reading is
still done using `zip-archive`. We might want to switch that over in a
separate PR.
* language: Check for different SDK versions when building pkg db
This fixes#2510. We check whether packages were compiled with differen
SDK versions and emmit an understandable error if so.
* addressing moritz comments
* Bump stackage
This PR switches us over to the latest Stackage LTS 0.14.1 (we were on
0.13.x before, so this includes major bumps) and gets rid of some
obsolete overwrites.
* Make stack traces work with cached top level values
For stack traces to work properly in the presence of cached top level values,
we need to cache the stack trace together with the value and restore the
stack trace when we get the value from the cache.
We also need to push location information under the monadic. Otherwise, the
location information will be removed from the continuation stack before the
monadic action is _executed_.
We also change one test case to use `fail` instead of `error` since `fail` is
more tricky than `error` (due to the lazy expression embedding in DAML-LF).
Unfortunately, the test did not work woth `fail` in the past because of the
issues fixed in this PR.
* Explain special cases when pushing location information
* language: append the version to the output dar name by default.
We now by default output foo-1.0.0.dar instead just foo.dar. Also the
maven coordinate default naming got removed.
* fixing integration tests and quickstart.dar occurences
The hie-core tests are flaky in stack. They can fail with the following
error message
```
hie-core: panic! (the 'impossible' happened)\n (GHC version 8.6.5 for x86_64-unknown-linux):
Dynamic linker not initialised
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
```
Explicitly initializing the dynamic linker at session startup should
avoid this issue.
This fixes all flakiness in `damlc test` that I was able to
reproduce. Previously, I got it to fail in about 10% of the cases
whereas now I have successfully run tests 200 times under load without
issues.
There were two issues at play here:
1. We run scenarios in separate threads to be able to kill the running
Shake session quickly even if a scenario has an infinite loop or
something like that (there is a timeout but it’s quite long). This
could result in one of those left-over threads trying to issue a
request while we are already trying to shut down.
To fix that, we wait for the concurrency semaphore to be empty before
shutting down.
2. Just waiting for scenario executions is not quite sufficient as
`runAction` does not wait for all rules to finish (we could just use
runActionSync in `damlc test` but I’d rather make this work
properly). While we do wait for all scenario executions to finish
there is one gRPC request in offInterest that we do not wait for:
gcCtxs.
To fix this, I’ve now routed all gRPC requests through the semaphore
which means that we will also wait for these requests to finish (or
prevent them from spawning).
This makes more sense anyway as scenario executions are mostly fairly
cheap requests while things like setting up the context are expensive
so we want to limit their concurrency.
We should make the concurrency limit configurable but I’ll leave that
for a separate PR.
* Extract template instance doc from newtype constructor doc
* Use new ghc-libs
* Remove type application which is not needed anymore
* Update daml-doc output with template instances
* Update hlint version
Apparently, we never had any tests for this, so this PR adds at least
a rudamentary test. The logic for this is rather stupid and easy to
break and might change soon so I’ll hold off on adding more extensive
tests until this works a bit better.
SS.scenarioServiceClient does not just read the actual client from
some IORef, it registers the available gRPC methods. Apparently we
never knew about this or at least I didn’t.
By only doing this once, we should speed things up a bit and this
fixes once of the assertion failures that we have been seing on
shutdown (pthread_mutex_lock(&mu->mutex) == 0 in sync_posix.cc) which
was caused by trying to register a method from another thread after
destroying the channel.
Add a command line option to `damlc inspect` which allows for configuring the
details level of the pretty printed DAML-LF. Right now the only difference is
that levels bigger than 0 print all location information. Level 0, which is
the default, also prints location information top level declarations.
This feature is useful for debugging location information.
* Show function names in stack trace on failing scenario
So far, we've only shown the location of the function but not its name.
Now, we add the name of the function as well.
* Only report progress when client supports it
This fixes an issue that some people encountered when running hie-core
in Emacs with a version of haskell-lsp that does not understand
progress events.
* Fix tests
* More test fixes
* language: fix: compute correct source root when building dar
Previously we just took the base directory of the main file, which is
wrong when the main is deeper down in the module structure.
* hlinting
* reuse moduleImportPaths
I assume the plan was to implement stack traces. I intend to do that as well
but the message type does not fit my approach. Thus, let's remove it first.
* Print stack traces in the scenario on failure
Currently, we only print the last source location, which is not
particularly helpful for debugging. Now, we put all source locations we
encounter during execution on the continuation stack and print them when
a scenario fails. This PR does not print the names of entered functions
or choices. We leave this for a future PR.
* Address Moritz' comments
Otherwise, we can end up retaining references to the old map which
prevent it from being garbage collected.
On a simple testcase that repeatedly opens and closes a module, this
seems to make memory usage constant whereas it was increasing
each time before.
* Add md_instances field to ModuleDoc
* Extract instances
* Add instance fields to class and type docs.
* Implement distributeInstanceDocs
* Use Maybe list instead of list.
* Render instance docs.
* Add instance data in tests and add a golden test.
* Rename to orphan typeclass instances.
* Upgrade ghc-libs supporting generic templates
* Update Proposal and ComposedKey tests to generic template syntax
* Temporarily patch daml-doc test output (will need to fix for template instances)
* Add exports to doc ctx
* Filter out unexported things.
* Added a golden test
* Fix missing constructors.
* Hide unexported class members
* Add rst golden test as well
* Remove unnecessary HIDEs from stdlib.
* lint
* Upgrade haskell-lsp and lsp-test
There have been some fixes upstream that should hopefully mean that we
no longer need to mark the lsp-tests as flaky on Windows. I am having
trouble reproducing the flakiness locally, so let’s see what happens
on CI.
* Also bump stack.yaml
Currently, we pass the flavour of the type constructor around although that
is cheaply computable from the data constructor itself. Thus, don't pass the
flavour around.
For every enum constructor `Foo` we produce a function `$ctor:Foo` whose
value is simply `Foo`. When we convert `Foo` to DAML-LF, we convert to
a call of `$ctor:Foo`. Since enum constructors are always fully applied,
this indirection is useless. Thus, we remove it in this PR.
GHC produces a function `$WFoo` for every record data constructor `Foo`.
The conversion to DAML-LF produces a function `$ctor:Foo` which does exactly
the same. We stop doing this and use the `$WFoo` function instead. Since GHC
does not produce `$WFoo` for newtypes, we need to produce these nevertheless.
I'll leave solving the corresponding issue for variant types to another PR.
Right now, the set of internal functions seems to depend on the DAML-LF
version although it actually doesn't. Since GHC is smart enough this doesn't
bite us but doesn't feel particularly clean either. If the internal functions
ever start to depend on the DAML-LF version again, we should probably put
them into the environment of the conversion to DAML-LF.
The code removed was responsible for injecting DAML-LF primitives for
`create`, `exercise`, etc. for the old template desugaring. These code
paths are not hit anymore with the new template desugaring.
* Only change project root once
As described in #2449, calling withProjectRoot' twice breaks with
relative paths and is also just silly so this PR fixes this by
factoring out the actual logic from init from execInit which does the
project root thingy.
* Add a regression test
* Avoid one indirection for dictionary construction
For every constructor `Foo` we generate a function `$ctor:Foo` which calls
the DAML-LF constructor for `Foo`. This function is useless since GHC already
creates a function `$WFoo` for us which does the same. This PR remove the
`$ctor:...` functions for type class dictionaries.
This makes a bit of a mess in the `Ctors` data type. This will be cleaned up
when all `$ctor:...` functions are finally gone.
* Fix construction of XInstance templates
This is a small step to solve the package name ambiguity problem.
Putting the package version into the package name allows to import a
package two times with different versions.
For example, we print two spaces if there's no annotation between `def` and
the function. This PR fixes this by using the `empty` document instead of the
document containing the empty string. (Please don't ask me why they are
different.)
Comparing FilePaths is really slow so by mapping them to Ints, we can
speed up dependency chasing significantly.
We might want to switch to doing some kind of global hash consing of
file paths at the Shake level but for now, this seems like a nice
improvement while not being too invasive.
This is roughly an ~8s speedup on my testcase.
Until now the test used the manually desugared syntax for the `Iou`
template. Now, we use the proper `template` syntax. The test is also
renamed from `IouDSL` to `ProposalIou` since the DSL would be confusing
after this change. In fact, we're mostly testing that the
instantiation of `Proposal a` with `a = Iou` actually works.
* Remove tests that are obsolete with new template desugaring
The tests were only relevant during the transition phase to the new
template desugaring. Since that desugaring has landed the test don't
provide value anymore and can be deleted.
* Fix super class dictionary detection
* Use tasty-golden for daml-doc's golden tests
With our hand rolled version of golden tests it is pretty painful to update
the golden files when they must change. After this PR it is as simple as
```
> bazel run //compiler/damlc/tests:daml-doc -- --accept
```
* Address indentation issues
* Use diff command line tool for diffing
* Removed new unused dependecy on Diff package
* Ignore carriage returns in diff
This allows us to GC the unsimplified DALF which decreases memory
usage and GC pressure. This does make GenerateRawDalf slightly slower
which could in theory have an effect on IDE performance since we use
the raw DALF for the scenario service. However, I haven’t been able to
measure any regressions (if it does become an issue, we could disable
optimizations completely in the IDE). In fact, things seem to be
slightly faster.
On my testcase max memory usage does seem to go down a bit but not a
lot (3.0GB to 2.7GB but it fluctuates somewhat between runs).
* Rename HaddockParse -> Extract
* Add --qualify-types option.
* More qualified type options
* Update help text
* Update compiler/damlc/lib/DA/Cli/Damlc/Command/Damldoc.hs
Co-Authored-By: Jost Berthold <jost.berthold@digitalasset.com>
* Accidentally left in a testing change.
* Documenting DocCtx fields
* Mark new options internal
* Fix daml build for usernames with spaces
This PR fixes two issues caused by having spaces in your username:
1. On Windows, we need to quote the path to the daml binary in the
batch wrapper (quotes are not valid in usernames, so no need to worry
about escaping them).
2. Invoking ghc-pkg via callCommand broke since shells are
terrible. Luckily, we can easily get away with just using callProcess
here.
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: associahedron <231829+associahedron@users.noreply.github.com>
This PR moves as much work as possible to GetLocatedImports which
contracry to GetDependencyInformation is shared between rules.
It’s still slower than it should be and somewhat messy but at least
it’s slightly faster and imho cleaner than before.
* Make --shake-profiling use the directory based profiling
The single-file based profiling is rather useless in the IDE and I
always found myself having to modify the source to set `profileDir` so
this PR switches the CLI option to control that instead.
* Add --shake-profiling to damlc ide
* Unify rst and md output in damldocs.
* Fix errors
* Define RenderUnwords / RenderIntercalate in terms of RenderConcat
* Delete the old
* Small output format changes and update golden tests
* Fix table rendering and tests.
* Fix golden test.
* Delete more old
* Implement reviewer suggestions
* Optimize mangleIdentifier
I am slightly embarassed by importing Data.Text.Internal and
Data.Text.Array. However, this does make a very measurable difference:
On my testcase it brought down the time encoding modules from 22s to
17s, so it’s a 5s improvement.
I did quickcheck this against the old implementation, so I’m confident
it behaves correctly. I also ran some criterion benchmarks to see the
difference and on its own it’s even more of a difference.
https://gist.github.com/cocreature/822114257227473ecff1638a88f07788
* GHC is stupid :(
* language: put hash in package directories
We put the package id of the main dalf of a package into the directory
names, where we store the files of that package in the package database.
This way we make sure that two equally named packages don't overwrite
their dependencies and files.
This is in preparation for #2326 as well as for splitting hie-core
into a separate repo. Given that, it explicitely avoids using our
dev-env.
We do need to install a few system packages, so for now this uses the
hosted builder so we can do this. Another option would be to just add
those to our builders. I don’t really have a preference either
way. The builds are < 5 minutes so I don’t expect issues from using
the hosted builders.
* Update ghc-libs to use new template desugaring
* Replace old template typeclasses with generic-friendly ones
* New template desugaring doc
* Fix tests
* Fix damldoc tests regarding Archive choice
* Update visualisation code to not traverse master dictionary
* Additional class method stubs to template instance declaration for upgrades
* Increase stack limit for bond trading compilation test
* Update hlint version
* Supporting producing sdist tarballs for the HS ledger bindings
The README.md has an explanation for how you can use this.
This should hopefully allow others to experiment with the bindings.
My benchmark is sadly extremely noisy which makes it hard to figure
out how much of an improvement this is but it does seem to at least be
a minor improvement.
* daml-lf: prepare archive proto for Numeric
* replace DECIMAL by NUMERIC in messages and fields
* add nat kind and nat type
* add builtins CAST_NUMERIC and SHIFT_NUMERIC
* daml-lf: remove new builtins from archive proto
* daml-lf: some more doc about numeric
* daml-lf: fix spec
Speed up file modification checks
Summary: `getModificationTime` from the `directory` package is really
slow. The `unix` package is faster but still slow. This PR brings the
time spent checking file modifications (which is required on every
change) from ~0.5s to ~0.15s.
* Add anchor self-links in Rst docs
* Refactor a little with unwords
* Fix silly type error
* Refactoring and better type rendering
* Rename rst renderer functions.
* Update tests
* lint
* language: a shake rule to get interface/hie files
This adds a shake rule to get module interfaces and hie files. This
gives more control on when to build them and also an opportunity to
change the package name after typechecking. This is used in the next
PR to add package hashes to the package name in the interface files.
* generate hie files only on demand
Currently, we call `force` on every result of a rule. This was
problematic for the rules that produce LF packages:
Even if only the top-level module changes, the package for that module
contains all its dependencies. This means that we end up RNFing every
module that the current module depends on. On a project with ~900
dependencies this added up to over 1s for both GeneratePackageDeps and
GeneratePackage which corresponded to more than half of the overall
runtime.
This PR introduces a newtype for packages that does not evaluate the
modules in a package to NF. With this PR both rules are below 0.02s so
his is quite a significant improvement. We do not introduce any
potential space leaks here since the modules are produced by rules
that already force them to NF.
* Improve debugging output
Displaying the exception makes it easier to figure out what is going
wrong.
I’ve also added a HasCallStack constraint to `locateRunfiles` since it
looked like that was failing. Turned out to be a call to `create` that
didn’t go via `locateRunfiles` but I think it’s useful either way.
Should be more useful with https://github.com/tweag/rules_haskell/pull/1007
* Update compiler/hie-core/src/Development/IDE/Core/Shake.hs
Co-Authored-By: Andreas Herrmann <42969706+aherrmann-da@users.noreply.github.com>
* Upgrade ghc-lib
* Patch bazel_tools : hazel-include-paths patch, no-isystem patch.
* Provide "haskell_c2hs" for package name to cabal_haskell_package
* Package name haskell_c2hs => c2hs.
* Switch to less hacky patch for include dirs
* damlc visual: detect ledger actions at surface level
Instead of detecting the DAML-LF primitives for `create` and `exercise`,
we detect the corresponding surface DAML functions and ignore the
dictionaries for the `Template` and `Choice` classes. This will be
necessary for swiching to the new template desugaring.
This is the quickest way to get this working but might turn into a brittle
maintenance nightmare one day. If that happens, we should switch to doing
proper symbolic evaluation, which will we be a bit more code though.
* Fix explanation
Co-Authored-By: Rohan Jacob-Rao <rohanjr@gmail.com>
* Add explanation for special casing archive
* Support depending on potentially stale values in damlc
For now, this is opt-in and only enabled for the scenario service.
Locations should be properly mapped so if lines are inserted above a
scenario, the scenario link will move down.
* Erase key lookup methods implemented using magic as well
We're already erasing `create`, `fetch` and `exercise` methods. I've
missed the ones related to key lookups when I did that. This PR fixes
my mishap.
* Fix typo in docs.
Co-Authored-By: Rohan Jacob-Rao <rohanjr@gmail.com>
We want to discourage users to pin down a target DAML-LF version but rather
want them to always use the default version, which is typically the newest
version. Thus, we turn this into an expert flag that is mostly used for
testing by us.
The module was only used to format decimal numbers. The only impact the
formatting could have had was to add/strip trailing zeros from the
presentation of the number. However, manual testing shows this did not
happen. Thus, the whole module is now useless.
`template X` generates type class `XInstance` which contains a few methods
which are implemented in terms of `magic` and then rewritten during the
conversion to DAML-LF. The default implementations of these methods need to
be erase during the conversion to DAML-LF since `magic` is erased as well.
* language: added a merge command to damlc
This adds a 'merge-dars' command to damlc to merge two dars into one.
This is useful when you created an upgrade dar and want to ship it as
part of the latest version of a package.
* added a comment on nubSort for clarity
* hie-core/test: Dedicated test suite for hie-core
* Apply hlint suggestion
Use System.Environment.Blank's setEnv which has a non-overwrite mode
that implements precisely what we were doing with more code before.
* buildifier fixes
* hie-core-exe works on Windows now - ghc-paths was fixed
This is a refactoring-only PR in preparation for supporting stale
values in damlc so we can still produce some results if a rule fails
to produce a value but has a stale value stored.
While experimenting with the pure Haskell gRPC implementation I
noticed that there is a fair amount of duplication in PrettyScenario
so this PR cleans up some of that.
The gRPC library does not handle asynchronous exceptions properly and
ends up leaking things which causes various issues (segfaults,
assertion failures, weird backup poller timer messages, …) when
shutting down gRPC.
The switch to grpcShutdownBlocking is somewhat unrelated (the issues
happen with and without that and the fix seems to work both times) but
that function seems to be the right way to shut down gRPC in newer
versions and is what all the official language bindings switched to,
so I also made the switch.
I haven’t yet looked into the issues in the HS ledger bindings so not
sure if this helps with those as well.
* language: upgrades independent of stdlib
This fixes several issues in the code generation from dalfs. As a result
we can now generate upgrades independent of the stdlib source.
Some `Development.IDE.*` modules were imported qualified as either `Compile`
or `CompilerService`. These names are at least odd and maybe also
misleading. Since there's no actual need to import them qualified, let's
just import them not qualified.
* handling dalfs flowing to next line
* space not blank
* Lints and space
* fixing the test names
* filter using not.null
* Using fold instead. makes reading easy
* Format everything
* Removing use of partial function
* More stdlib tweaks
* Another commit to kick off build again
* Fix incorrect docs syntax
* Apply suggestions from code review
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* Apply suggestions from code review
Co-Authored-By: Martin Huschenbett <martin.huschenbett@posteo.me>
* More suggestions applied
* Support reading of DAML-LF 1.5 in damlc again
We dropped support for this for no good reason when we dropped support for
writing DAML-LF 1.5. Being able to _read_ older versions of DAML-LF is
important for the upgrading story.
We don't support reading older versions of DAML-LF since there seems to
be no need for this.
* Fix integration test
* DA.Bazel.Runfiles based on bazel-runfiles
* locateRunfilesMb -> locateRunfiles
* .exe extension on Windows
* Add docstring to locateRunfiles
* bazel-runfiles: Normalize on Windows
* language: correct translation of sums of products in daml code gen
We fix the translation of data type constructors with several name
components for sums of products in the translation from daml-lf back to
source.
There is lots of room for improvements here but I think this is a good
first step. The 3 main things that could be improved imho are:
- Rewrite source locations to point to the original file rather than
the generated module
- Provide some way to declare things like imports or more general,
setup code that is added to the generated module.
- Prettier/more helpful output during a run, e.g., print the list of
successful tests.
* internal name
* Adding internal choice name
* correct but ugly
* no more awkard underscore for create and archive
* not more underscores
* creating archive is not necessary
* simplify
* Removing duplicate
* simplified code
* Using cons intead of ++
* not needed parans
* Formatting
* Use type information in function docs
* idpToText -> packRdrName
* Use tycon to get superclasses
* Make anchors from tycons
* Generate all type anchors with tyConAnchor
* Type information is everywhere
* Render links.
* s/missing/available/
* Update golden tests
* Hlint / refactor
* Clean up import list
* Banish evil
* Catch lint
* Use mapMaybe
* Use list comprehensions instead
* packModule -> getModulename, dc_mod -> dc_modname
* Deal with tuple constructors properly
* Update the test case
* Better docs in Rst renderer
* Add test that uses type sig links
* Update release notes
* Get draft rule in
* Hlint is now firing
* First hlint unit test
* This gets hlint working in the ide
* Select hlint.yaml contextually. Wip
* Fix prod path
* Switch between prod and test versions of hlint.yaml
* Format
* Ignore hlint hints in the language tests
* Add some debugging stuff. LSP stress test is broken
* Update compiler/damlc/daml-ide-core/src/Development/IDE/Core/Rules/Daml.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Tidy up some details
* Simplify settings (remove unused parseFlags)
* Fix LSP stress tests
* ./fmt.sh
* Collapse to one hlint.yaml, provide options
* Fix location of hlint runfiles
* Disable hlint by default; enable for shake-ide tests
* lint (ironic)
* Remove filtering of lints from integration test
* ./fmt.sh
* Respect the flag
* Silence opininated lint
* Update compiler/damlc/daml-opts/daml-opts-types/DA/Daml/Options/Types.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* Update compiler/damlc/lib/DA/Cli/Damlc.hs
Co-Authored-By: Moritz Kiefer <moritz.kiefer@purelyfunctional.org>
* I really did mean optHlintEnable here
* Move hlint enabled check to ofInterest like shouldRunScenarios
* Rename variable
* Cache settings
* Missed a merge conflict
* lift liftIO
* Remove the hlint-testing POC
* use useNoFile_
we make sure the right things are qualified and the right things are not
qualified in generated code. we also add GHC.Types always to the imports
because it's potentially used by native types such as 'Int' and we don't
import Prelude.
This adds two compiler options ifaceDir and isGenerated to the compiler
options. These will be needed to write generated interface files to the
right directory and treat generated source files special when generating
interface files.
There were two issues:
1. VSCode changed their packaging so we now need to depend on
`@types/vscode`. This broke `yarn run compile`. I have no idea why
this did not cause an error in the Bazel build. Presumably the giant
tsconfig that it generates was doing something different.
2. We excluded src/webview.js in .vscodeignore which means that it was
not included in `vsce package` and thereby also not in the extension
included in the marketplace.
For now, we only show a “Progressing” message and a done/todo
indicator. We could eventually try to come up with something better
but I’m not quite sure what that would be since we try a lot of things
in parallel and the triggering request isn’t particularly
useful (users won’t know what a codelens request is and why they have
to wait for it).
Note that VSCode seems to have some delay in updating these
notifications so you only see the done/todo reports if it is
processing for a while.
* Run typechecker in damldocs (but don't use result yet)
* Add Modulename to TypeApp constructor.
* Add md_name where appropriate.
* Thread a DocCtx through each doc extractor.
* Move template set and choices map into docctx
* Use Typename for template set and choice map
* Make Anchor a newtype and move it to Types
* Make Modulename in TypeApp an Anchor instead.
* Remove import of hidden module.
* Qualify that which doth need be qualified
* Update tests
* Build stdlib in such a way that it passes typechecking.
* Update release notes.
* Remove MOVE Prelude directives in daml-prim.
* Use optional in damldoc options
* Update function hashes
* Compile base packages to json first
* Proof of concept : hlint/daml
* Copyright check is naieve
* Missing copyright
* Move hlint-testing out of compiler
fmt.sh tries to invoke hlint on the compiler directory. This then fails like,
```
2019-07-08T21:04:48.8037836Z $ hlint --git -j4
2019-07-08T21:04:48.9217137Z Couldn't find file: hlint-testing/data/HLint_QuickCheck.hs
2019-07-08T21:04:48.9277694Z UNEXPECTED HLINT FAILURE: The Bazel rules should have spotted this, please raise a GitHub issue
```
* Reformat
* This linting and formatting and so on is getting tiresome...
* Replace links with real files
* Strip the data directory
* Move files back under compiler/hlint-testing
* Copyright report_template.html so that format checks pass
* Calculate a hlint data directory path
* Whitespace and delete unneccessary file
* The format script uses this
* Use locateRunfiles
* Tweak record initialization formatting
* Reformat build file
Previously, we unconditionally assumed that GHC will only use those on
Enum-like types, i.e., types with only nullary constructors. This
assumption turns out to be incorrect so this PR introduces additional
checks.
* Move files in daml-foundations/daml-ghc to compiler/damlc
There is also a bit of refactoring going on to actually split things
apart into sensible targets. What is still missing is a cleanup of the
module hierarchy and a cleanup of the test targets but I’ll leave
those for separate PRs.
As a nice bonus, this also reduces dependencies between targets so it
will speed up compiles.
* Update .hie-bios
Previously we rewrote references to `PRSelf` to `PRImport` references
every time we call `initWorld`. However, we call `initWorld` quite
often (e.g. every time we run a scenario) while the packages that need
to be rewritten stay constant. Since rewriting the package references
requires traversing the whole package, this can be quite expensive.
This PR moves the rewriting of package references out of `initWorld`
and caches it as part of `GeneratePackageMap`.
On a large project where I tested this, this caused a drop in the
runtime of `daml test` from 250s to 200s. In the IDE this can also
make a pretty big difference since we call this everytime we run a
scenario which we do on every file change.
* check that submitter is in maintainers when looking up keys
Fixes#1866. Note that this limitation applies both for `lookupByKey`
and `fetchByKey` -- anything involving retrieving a key is affected.
* add UNTIL-LF to run tests up to a certain version of DAML-LF
* name targets for DAML tests better
* add notes about DAML-LF changes
* commit Test.daml with DAML-LF 1.5 rather than compiling it on the fly
* add scenario tests for #1866
* add warnings about future key behavior in docs
* use flag rather than version when executing
Most of them were caused by off-by-one errors in goto definition.
There was also one test that was marked as an expected failure but the
actual bug has been fixed for some time and the only reason it was
failing is that the error message was different than the test
expected.
I’ve also renamed daml-ghc-shake-test-ci to damlc-shake-tests which is
something that I might actually be able to remember :)
Previously, we only checked two diagnostics messages. However, the
second diagnostics messages does not actually need to correspond to
the latest change but can just be a message that has been in the queue
from one of the 1000 changes before. Now we make sure to actually
check all (or at least every second) diagnostic.
This was technically also an issue before but since we only emitted
diagnostics when they changed and we alternate between two states it
worked fine. The way we implement debouncing means that we can now
also end up emitting two consecutive diagnostics changes for the same
set of diagnostics which made this test flaky.
Given that we already made the max message size configurable it only
seems reasonable to also make the timeout configurable and on very
large projects, we do sometimes hit this.
Previously, we emitted diagnostics notifications as soon as we got
them. This resulted in a lot of flickering due to diagnostics getting
cleared briefly when typing only to immediately reappear.
Now, we buffer them for 0.1s so that a new event restoring the
same diagnostics for a slightly modified file will overwrite the
initial clear of diagnostics for the new document version.
* Split off addRelativeImport from modifying the session
* Switch located imports to adding their own relative imports
* Delete the unused getGhcDynFlags
* Make runGhcEnv no longer need to consult import paths
* Call runGhcEnv directly
* Move getSrcSpanInfos out to Spans
* Remove a redundant import
* Make findImports in Either rather than ExceptT
* Move getImports over to the right place
* Switch to liftEither
* Get rid of orphan usage in Convert
* Get rid of orphan usage in AtPoint
* Delete one entirely unused orphan
* Remove a comment that is no longer true
* Add a .gitignore just for the hie-core piece
* Add standalone pieces to make hie-core work with a global Cabal install
* Add more things to .gitignore, PR suggestion
* Add copyright header
* Allow shadowing of type variables in DAML and DAML-LF
We relax the DAML-LF type checker to allow for shadowing of type variables.
This does not need big changes since the substitution we use already avoids
name capture. The test for alpha equivalence converts to de Bruijn indices
on the fly and is hence not a problem either.
We inline type synonyms during the conversion from GHC Core to DAML-LF. This
requires alpha renaming. The cheapest way to get this, is to use the unique
names of type variables instead of their surface names.
This fixes#1915.
* Fix Scala test
* Relax memory constraints for bond-trading test
Previously we sometimes ended up declaring the expected exit of the
scenario service as an unexpected exit. This PR addresses this by
adding a new variable that we set to True before we ask the scenario
service to exit by closing stdin.
* Allow controlling the gRPC message limit via daml.yaml
We have had to raise that in the past since it caused issues on large
projects so it makes sense to make it configurable.
* Update unreleased.rst
Co-Authored-By: Beth Aitman <bethaitman@users.noreply.github.com>
* Fix running the IDE on damlc
There were two issues:
1. Missing include paths.
2. Files where the module name does not match the file name.
I’ve fixed both and added a test that we can load the damlc Main.hs.
* Update rules_haskell and static GHC
Remove patches that have been upstreamed or are no longer required.
Update still required patches to match the new rules_haskell version.
Previously we patched rules_haskell to coerce GHC into using static
Haskell libraries in most places. In particular we moved hs-libraries
entries into extra-libraries entries in the package configuration files.
A much cleaner approach is to compile GHC with a static RTS, then GHC
will by itself choose to load static Haskell libraries.
* Remove haskell_cc_import
* da-hs-daml-cli -> daml-cli
* da-hs-damlc-app -> damlc-app
* Push the suggestion work further up
* Make LspFuncs an argument to the handlers
* Actually pass around the contents of the buffer to suggestAction
* Make suggestAction do sensible figuring out if you remove the next line too
* Better indentation
* Code action to add GHC extensions as required
* Deal with extra arguments to LSP handler
* Add a code action
* Update comments
* Remove logging, since its too verbose
* Fix a few warnings, add a final case
* Add an example of what the code action matches
* Add more comments to shakeRun
* Delete the multiple versions of runActions, since they weren't used and parallel is good enough
* Delete runActionsSync entirely
* Make sure runAction returns even if shakeRun throws an exception
* Remove the callback from shakeRun - it was never used
* Fix one last use
* More comments
* Rename reportSeriousError to reportInternalError
* Stop using logError for logging things that are warnings to the user, not errors by us
* Rename logError
* Sort the log fields properly
* Delete tagAction from Logger
* Strip down the pure logger
* Delete unused pieces of the logger
* A quick check suggests the call stack will be useful in approximately none of the callers of logging, so just remove it
* When reporting an internal error, give as much detail as we can
* Change our logger to be based on Priority values
* HLint fixes
* Rename makeNopLogger
* In hie-core say what level of message you are setting
* Delete the unused makeOneLogger
* Make sure we can show messages floating around
* If a notification/response handler throws an exception, report it upwards
* Remove reportInternalError in favour of a general logging mechanism
* Add missing dependencies
* Just call fail for a dodgy error report
* Add a FIXME
* Make missing modules just an error
* Pull the CPP into a separate module
* Pass Nothing to indicate that a text buffer shoud just be used from disk
* Add save handlers, since the version changing to ModTime may have an impact
* Rename contents to mbContents in one place
* Change runCpp to take a Maybe StringBuffer and attempt to reuse the existing file, if it can
* Add a Bazel alias for hie-core
* Add notes about the sad path
* Avoid one use of filePathToUri
* Avoid another use of filePathToUri which went wrong for CPP output
* Normalize Uri's by replacing adjacent // with a single /
* Improve how CPP works if you have a modified buffer
* Move textToStringBuffer out to Util
* Switch to hPutStringBuffer which is in GHC 8.8
* Note why we are escaping to /
* Refactoring suggested by review
* Move the hie-core demo files around (they aren't really a demo anymore)
* Split the command line parsing into a separate module
* Give messages about how long starting something takes
* Make the interactive mode say what it is doing a bit more
* Add a --cwd flag to hie-core
* Take a list of files and directories for hie-core
* Update the readme to say how to test using hie-core
* Fix up the bazel file
* Add HLint exception
* specify the new fields for interning package IDs
* percolating intern tables
* crawl packages for package IDs as a pre-phase: generic prep
* stub case for interned_id in Scala packageref reader
* HasPackageRefs instances for the rest of the ast
* make intern table and use when encoding PackageRefs in v1
* don't need where
* stub out decode for interned package IDs
* no benefit to using uint32 instead of uint64
* percolate in encode one step
* interned case for decoding PackageRefs
* naming details
* intern table decoder
* finish propagating the intern table in encoder
* encode the package ID table
* document the vital assumption of encodeInternedPackageIds
* propagate the intern table through the LF decoder
- done by stacking ReaderT on top of Decode internally,
as discussed with @hurryabit
* daml-lf-proto requires mtl
* stub out interned case in Scala LF decoder
* stub interface decoder function
* get the interned table to most places in InterfaceReader
* support for interned package IDs in Scala decoder
* use ImmArraySeq instead of Vector for Scala intern decode table
* adding that ghc extension didn't make sense
* implement interned ID decoding for InterfaceReader
* scenario service won't have interned package IDs
* test the interned ID resolution in Scala by examining the proto -> AST in detail
* proper precondition for the dev phase of interned IDs testing
* better error reporting for malformed DALFs in intern test
* just import Data.Int
- suggested by @neil-da; thanks
* pass around the lookup function instead of the vector in decoder
- suggested by @neil-da; thanks
* remove derivations for types deleted in e63b012d2d
* rename VersionAware to EncodeCtx
- suggested by @hurryabit; thanks
* rename MDecode to MonadDecode
- suggested by @hurryabit; thanks
* pass a function through the encoder instead of a set
- based on suggestions by @hurryabit and @neil-da; thanks
* daml-ghc test that interned IDs are generated
- suggested by @hurryabit; thanks
* adapt to 5b480c99ec#1844
* Move mergeHandlers into Server
* Make partial handlers a proper newtype
* Pass the options in to runLanguageServer
* Take in user handlers
* Remove the code lens handler since we don't advertise it
* Add setSomethingModified, rather than faking it for the LSP VFS
* Rewrite the LSP notifications
* Improve the display of info messages around openning and modifying text documents
* Make sure stdout and stderr don't have buffering, so we see their output immediately
* Handle exit properly
* Make notifications forward on to their previous values
* Remove the exit handler, HIE already has a good default for it
* Add comments on FileStore
language: upgrades: dont derive generics for data types having the instances already
We only derive generic instances on the fly for data types that don't
have them already.
* Inline chunks of LSP.Server into LanguageServer
* Inline runServer
* Start figuring out a better API for gotoDefinition
* Remove old JIRA ticket numbers
* Add a hover handler in the new form
* Change the new handlers slightly
* Add a new module to handle notifications updating the virtual file system
* Rewrite the language server in hie-core to use the Handler more directly
* Add a cancel handler
* Ignore a few more handlers
* HLint
* REname functions that set handlers
* Rename a few more set handlers
* Delete the unused makeResponse
* Move mergeHandlers over to LanguageServer
* Rename RunHandler to WithMessage
* Switch from STM to IO
* Avoid the Protocol module
* Rename AddItem to Message
* Document why we use clientMsgChan
* Add comments around Message
* Delete unused package dependencies
* Eliminate the use of pretty
* Don't export some things that weren't necessary
* Add a missing dependency
* Avoid reexporting files of interest methods
* Put toIdeResult back
* add scenario Module decoding to Decode.OfPackage
* use purely data-driven decoding in scenario service in Scala
- decouples scenario service from LF decoder implementation
* make DecodeV1 companion private
* make extension to LFv2 more obvious
* Move prettyPosition over to a more sensible place
* Avoid some pretty printing
* Remove duplicate methods and switch to having a function to get the logger handle
* Remove unncessary bits of the logger
* Remove reportSeriousErrorDie - was unused
* Rename the Logger methods to Logger rather than Handle
* Delete the unique supply
* Decrease the use of getServiceEnv
* Move getIdeOptions inside Service
* Add getFilesOfInterest rule
* Hide the existence of Env
* Inline some redundant forwarding methods
* Add a FIXME for a dodgy function
* Delete some redundant imports
* Rename Map to HMap, since I need to use Data.Map too
* Make the Shake-specific Diagnostics functions into that module
* Delete errorDiag which was unused
* Inline the diagnostic function, since it had one user
* Add ofInterest rule as a separate module
* Sort the exposed modules
* Fix up the demo
* Make sure you add the OfInterestVar global
* HLint
* Get rid of some of interest stuff in other places
* Remove the OfInterest stuff from Service
* Delete LoadPackageResult, was unused
* Delete tmrOccEnvName from Compile
* Push orphan instances around a bit, avoiding some
* Make convertModule take an explicit filename to report against (since we have a good one in our hands)
* Get rid of GhcModule - only one field was ever used
* Collapse setVritualFileContents and removeVirtualFile into one
* Make the VFSHandle abstract
* Make it clear runGhcSession does not need IdeOptions
* Avoid passing around IdeOptions so much when they aren't required
* Get rid of runGhcSessionExcept
* Make catchSrcErrors be in Either, not ExceptT
* Don't import ExceptT qualified
* Don't import Exception qualified
* Rewrite and simplify computePackageDeps
* Delete the entirely unused reflow function
* Don't have Data.Text.Prettyprint.Doc.Syntax reexport stuff
* Move most of the pretty printing inside hie-core
* Get rid of the prettyprinter-syntax dependency from hie-core
* Get rid of prettyprinter-syntax, by collapsing the one definition into DA.Pretty
* Update the hie-bios commit SHA
* Also update the SHA in our bazel WORKSPACE
* Update the hash too
* Tutorial for Emacs integration
* Update hie-bios patch
My primary goal is to avoid confusion when starting to support DAML-LF's
enum types. I also think these types were never particularly useful,
although I introduced them myself...
* Move the span related functionality to one place
* Fuse docHeaders away
* Decouple AtPoint from the rest of the rule database, simplifying the dependencies
* Move the import related functionality to one place
* Move all the closely tied to GHC modules together
* Rename the Logger module
* Push the other module renames through the code base
* Rename Development.IDE.State to Development.IDE.Core
* Rename Functions.Compile to Core.Compile
* Fix up some module names
* Cut down on non-sensicle exports
* Don't worry about setting source - no one uses it
* Reorder the module header
* Give more sensible names to the diagnostic creating functions
* Use more appropriate diagnostic functions
* Simplify the internal diagnostic creations
* Rewrite the diagnostics to go direct, not via GHC error types
* Remove redundant dflags from some functions
* Make sure the warning vs error distinction remains
* Remove unnecessary extensions
* Make atPoint generate either haskell or daml syntax, depending on options
* Get rid of data HoverText
* Move VirtualResource out from hie-core
* Rename hie-core to the right name
* Drop Types.LSP, merge it into LSP.Protocol
* Remove orphans that aren't actually used in our code
* Delete redundant newlines
* Add a defaultIdeOptions function
* Move ideTryIOException over to its one user
* Delete unused exports
* Delete the ProjectDiagnostic newtype - the type index was always Key
* Don't reexport position information from Diagnostic
* Delete the unused bits from Location
* Delete unused stuff from getSrcSpan
* Move URL stuff into Location from Diagnostic
* Remove unnecessary CPP
* Remove unnecessary extension
* Change from stage being polymorphic to being a Text
* Fix up the test suite too
* Push the cleanup to the edges
* More dependencies
* Patch up the tests now they need to find the type signature inside a ```daml block
Previously, we got the package id by listing all packages and taking
the first. That is not a valid assumption and broke with a PR. This PR
changes the tests to use the DAR reader that we already used for
visualization (now factored into a separate library) and reads the
main dalf using that.
* Move IDE stress test to lsp-tests
* Lower timeout.
The test case takes < 3 seconds on my machine. The 180 seconds timeout
seems far too much.
* Data.Text.Extended -> Data.Text
Handle is kind of a mess at the moment, this is a first step towards
cleaning it up:
1. We had two functions called getDalfDependencies with one wrapping
the other. This PR merges them into one to make this less confusing.
2. buildDar called runAction a bunch of times directly and via other
functions. This PR switches it to use a single call to runAction.
3. The logic for turning a Maybe into an `ExceptT [FileDiagnostic]`
was duplicated in various places. This PR factor out the logic into a
single function.
There is certainly more cleanup to be done (e.g., I don’t think
ExceptT buys us anything here, that module should probably die
completely with the logic being moved to other modules, …) but I’d
like to do it incrementally.
Previously we had a weird mix of using ExceptT to shortcircuit on
failed dependencies where the list of diagnostics was always empty and
throwing BadDependency exceptions.
This PR switches everything over to use BadDependency for failed
dependencies and removes a lot of conversions from one style to the
other.
I find the current names very confusing and unintuitive.
I'm aware that there are other primitives called `TO_TEXT_X`. But there the
`X` is meant to be a type index to `TO_TEXT`. This doesn't make much sense in
the context of `TO_TEXT_CODE_POINTS` though since there's no type named
`CodePoints`.
Previously, we always reverted back to the table view when the
scenario results changed. This PR changes this to preserve the
selected view.
I’ve also tested that this does not break backwards compatibility: If
you use the newer extension with an older SDK, you will get the
previous behavior of reverting to the table view and you can still
switch by clicking on the button.
Fixes#1675
* Make runActions return Return as soon as results are available
Previously, we were waiting for all rules to finish, in particular the
ofInterestRule. That doesn’t really make any sense, e.g., a goto
definition request should not be waiting for all scenarios to run.
The next step will be to change the LSP side such that requests and
notifications are processed in parallel where possible.
* language: feature: initial implementation of a 'migrate' command
We add a 'migrate' command to daml assistant that generates a project
that allows to migrate contract instances from package1 to package2.
This first version reads both package1 and package2 from source. As a
next step we read only the dalfs from package1, because it might have
been created with a different compiler.
This is a temporary fix to unblock users that have run into this
limit. The long-term solution here is to use compression and/or make
the limit configurable.
* Add more package.json attributes
* Rename ide-demo to haskell-ide-core
* Change the default flags for the IDE
* Update the README with how to install things
* Move getFilesOfInterest into the IDE, and OfInterest out into DAML
* Have the IDE mode kick all active files each time
* Add a missing .cabal dependency
* Add a missing dependencies
* Use the right thing, if any files have parse errors, then don't worry
* Print messages to stderr rather than stdout
* Add a big warning that writing to stdout is a bad plan
* Move Definition and Hover over to IDE.LSP
* Copy LanguageServer over to IDE Core, not ideal, but hard to abstract right now
* Warn that there are two copies of a given module
* Move printing out the SDK version out of language-server
For a cyclic import we actually have to write the files to disks,
otherwise we can end up with an error about the module not existing if
we check imports before the other module has been added to the VFS.
This implements part 2 of #1507 and fixes the daml-ghc-test-dev test
suite on Windows (not enabled on CI due to GRPC issues).
I have also tested this in the IDE on Windows and Linux.