As multiple platforms will create different annotated tags (because an
annotated tag includes a tag time), they will conflict on trying to
push. Therefore, we go for a lightweight tag for now, as those are
simple pointers to a commit and git will recognize that they are the
same and there is no conflict.
Warm up local caches by building dev-env and current daml master This is
allowed to fail, as we still want to have CI machines around, even when
their caches are only warmed up halfway.
Afterwards, we purge old agents that might still be around, that didn't
unregister themselves
This depends on #402 to be merged, as otherwise purge_old_agents.py
can't be found obviously.
* Accept multiple files in damlc test
Since damlc test also runs tests in transitive dependencies, this can
be significantly faster than running "damlc test" individually on
a set of files as you will end up recompiling and rerunning tests
multiple times if those files depend on each other.
For //docs:daml-ref-daml-test This is roughly a 10x improvement from
~70s to ~7s.
This rewrites the release script to be a lot simpler and significantly
faster:
- The artifacts are now declared in a separate yaml file which should
make it easier for people to modify and doesn’t clutter the actual
code.
- There is only a constant number of calls to Bazel which speeds up
the script quite a bit.
I verified that the release artifacts are the same that we got
before and I traced the calls to the jfrog binary in a fake release
and ignoring order they are identical.
This adds `ci/azure-cleanup`, containing a script that talks to azure pipelines, removing agents older than 25 hours in a specific pool.
Machines are meant to be killed after 24 hours anyway, make sure they're properly unregistered from Azure Pipelines, too.
By doing this, we don't need to unregister nodes manually on shutdown.
Idea is to execute this every time a new agent is provisioned, it has cloned the repo. We intend to clone the repo and pre-warm the caches there anyhow.
WIP until the repo fetching and cache pre-warming is present, too.
cc @zimbatm
### Pull Request Checklist
- [x] Read and understand the [contribution guidelines](https://github.com/digital-asset/daml/blob/master/CONTRIBUTING.md)
- [x] Include appropriate tests
- [x] Set a descriptive title and thorough description
- [x] Add a reference to the [issue this PR will solve](https://github.com/digital-asset/daml/issues), if appropriate
- [x] Add a line to the [release notes](https://github.com/digital-asset/daml/blob/master/docs/source/support/release-notes.rst), if appropriate
NOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with `/AzurePipelines run` to
trigger the build.
* Add buildifier targets.
The tool allows to check and format BUILD files in the repo.
To check if files are well formatted, run:
bazel run //:buildifier
To fix badly-formatted files run:
bazel run //:buildifier-fix
* Cleanup dade-copyright-headers formatting.
* Fix dade-copyright-headers on files with just the copyright.
* Run buildifier automatically on CI via 'fmt.sh'.
* Reformat all BUILD files with buildifier.
Excludes autogenerated Bazel files.
Azure Pipelines has direct integration with GitHub, so we're just using
that. Releases on GitHub have to target a tag, so we also need to push
the tag as an intermediate step; we also need to include the platform
name in the artifact to avoid overwriting from different builds.
The two "GitHub release" steps depend on two Azure variables that are
not defined in the pipeline script. This may look like it should not
work, but in fact it does, because these variables are set by the
release script.
In Azure Pipelines, any build step can set variables for the next build
steps by outputting specially-formatted text to stdout. This text will
not appear in the build output displayed by Azure Pipelines, e.g.:
```
echo '##vso[task.setvariable variable=sauce]tomatoes'
```
would define the Azure variable `sauce` to have the string `tomatoes` as
its value for the next build steps.
See [0] for details.
[0]: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#set-in-script
* TransactionServiceIT passes
* fixup
* using record time taken from TimeProvider
* taming TransactionBackPressureIT to avoid overloading of submission service
* implement AutoClosable on all Ledger related components having stateful resources
* reenabling InMemory fixture
* disabling contract key integration test for SandboxSQL fixture
* removing TODO
* bumping up timeout on TransactionServiceIT due to slow Azure pipeline
* 1 minute timeout for SqlLedgerSpec
* making jdbcurl CLI argument hidden
* updating release notes
* Initial rattle prototype
* Build the IDE core
* Ignore the Rattle directory
* Clean up the dependencies
* Require stack-1.10, since that does much better extra-dep caching
* Streamline the ghc-lib dependencies
* Compile more pieces
* Add a build.bat
* Make the Windows build use the correct stack.yaml to bootstrap
* Fix up enough to build on Windows
* Generate the dylib's on Mac
* Remove accidental -v
* Make the Haskell build driven by the Bazel metadata
* Get proto3-suite building
* Delete the unneeded haskell-dependencies
* Allow generating the proto files and compiling them
* Fix metadata to deal with """ syntax
* Fix metadata to deal with a list of globs
* More work in the direction of daml-ghc
* Use correct daml_lf proto version
* Tell GHC to use shared objects in TH
* Specify needed packages
* wip
* wip
* Switch to the fork of gRPC-haskell
* Build executables with rattle
* setup build.sbt in daml-lf
* Build binaries with rattle
* rattle-sbt, move scala build scripts out of daml-lf subdir, and into rattle subdir
* convert scala-build.sh into MainScala.hs
* Clean up rattle build
* Pre-merge clean up
* Switch to the newer version of ghc-lib-parser
* remove dev ls from MainScala.hs
* compile java generated from protos as separate projects
* Add copyright headers
* HLint fixes
* Uscrewup an HLint fix
* fix scala formatting of rattle/build.sbt
* Move tuple repacking in conversion to DAML-LF into separate function
I need to use the same logic for implementing the `TextMap.toList` function.
* Improve naming and add documentation
* Add location information to DAML-LF produced by damlc
This is required to get error locations in the scenario view. Rigth now,
the location information for `create`/`exercise` still points to the
template/choice. I'll fix that in a separate PR.
* Fix test expectations
* Fix more tests
The `daml-assistant` BUILD file was missing an `src_strip_prefix` field
and abtracted away the dependency list for two of its targets. The
latter is not strictly needed and may lead to too many dependencies
being specified for the targets.
* Add participant-state and participant-state-index APIs and reference implementations
This adds the (still WIP) interfaces and the in-memory reference implementations
of participant-state and participant-state-index.
See issue #137.
* Pass ledgerId through to getLedgerEnd method
This is needed in the daml-on-x implementation of the
transaction service.
* Add api-server-damlonx and the reference server
This is the initial version of the ledger-api server built on top of
the participant state APIs.
While functionally complete (modulo test services), it has only yet
been tested with the semantic tester. The server and the participant
state APIs are still under active development.
See issue #137.
* Apply scalafmt and copyright headers to new damlonx code
* ledger/damlonx: Fix build errors after merge from master
* ledger/participant-state: separate out the reference impl
* ledger/participant-state-index: separate out reference impl
* ledger/damlonx: Refactoring
- Refactor participant-state into multiple modules
- Introduce structured offset and update id types (vector of ints)
- Properly parse ledger feature flags
* ledger: Disable failing semantic test. Decrease grouping duration.
* scalafmt, fix after merge
* Add missing copyright headers
* ledger/damlonx: Add Simon's comments
* ledger/damlonx: Remove UpdateId, use Offset in both state and state-index.
* ledger/damlonx: scalafmt
* damlonxserver: review participant-state interface
Includes adding issue links for postponed fixes.
* daml-on-x-server: review reference implementation
Fixme's added to https://github.com/digital-asset/daml/pull/348 and
extra issues creaed in https://github.com/digital-asset/daml/milestone/4
* ledger/damlonx: Use SimpleString
* ledgre/damlonx: Backtrack on SimpleString change
See issue #398
* daml-on-x-server: drop unused 'index.impl.reference.package.scala'
* daml-on-x-server: add note on potential transient contracts bug
* ledger/participant-state*: post-merge fixes
* Add client_server_test bazel rule
This adds a generic mechanism for constructing a test that launches
a TCP server and runs a test-suite against it. The rule orchestrates
passing the port number from the server to the client via a temporary
file and takes care of killing the server when the client process exits.
Still to be done is figuring out a good way to pass additional arguments
to the server and client programs (where these arguments could be other
bazel rules).
* Add missing copyright headers
* Add support for arguments to client_server_test
Note that this does not support passing in labels (":my_data_file"),
nor does it support "$(location :my_data_file)" string macros.
* Add data attribute to client_server_test
* Add support for location expansion in client_server_test args
* client_server_test: Address code review
* client_server_test: Bump waiting for port file write to 5s
* Do not divulge contracts to observers in nonconsuming exercises
Disables support for non-default ledger feature flags, as they
are meaningless since ledger server logic does not respect the flags.
Instead of large refactoring to add support for the old flag settings,
it is best to disallow the deprecated flags, and later on phase out the
flags completely.
Re-enables test_divulgence_of_token in sandbox semantic tests.
Fixes#157.
* purge LedgerFlags entirely...
...since we only support one version of them anyway, and clearing them
* updated release notes
* Fix#152 java-codegen: Document parameterized types.
Explain the sources that the Java Code Generator creates for
parameterized DAML types including the implications for converting
between Java Bindings Value types and Java native types.
* Turn off -Woverlowed-literals in damlc
This flag does not play well with location information obtained via `-ticky`.
Also, the error message you get from overflowed literals suggests to use
`-XNegativeLiterals`, which is a bad idea since it changes the meaning of
`(-1)` from `\x -> x - 1` to `negate 1`.
* Fix module name in test
Co-Authored-By: martin-drhu-da <31696042+martin-drhu-da@users.noreply.github.com>
* HOTFIX damlc: make ghc-options public, refactor
* refactor `--ghc-options` again to support repetition and space-separated options
With this change, it is possible to repeat `--ghc-option bla`, and also to space-separate
multiple ghc options in one instance, like `--ghc-options "--flag1 -flag2 argument"`
