daml/bazel_tools
Andreas Herrmann e8001fc4ff
Speed up ghc-lib(-parser) sdist generation (#14076)
* Update ghc-lib to speed up sdist generation

To incorporate https://github.com/digital-asset/ghc-lib/pull/379, which
is a more generic version of the changes introduced in
41ab1c2cba
to speed-up ghc-lib(-parser) sdist generation in Daml.

The slowest step in the ghc-lib(-parser) sdist generation is the
generation of `.hs` files from `.hsc` files via `hsc2hs` and from `.x`
or `.y` files via `alex` or `happy`. The reason that it's slow is that
`ghc-lib-gen` performs these through GHC's build system, hadrian, and
these steps require almost a full stage1 GHC build.

The `.hs` files are only needed to enable dependency discovery through
`ghc -M`, as it doesn't understand `.hsc|.x|.y` files. Apart from that
we can use the original `.hsc|.x|.y` files in the final sdist.

With this update `ghc-lib-gen` finds all relevant `.hsc|.x|.y` files
and replaces them with dummy `.hs` files that have the same module name
and the same imports. These dummy files are only used for the purposes
of dependency discovery via `ghc -M` and are not included in the final
sdist.

With this update the sdist generation is sped up between 4.3 to 4.5
times:
- ghc-lib-parser: 3m2s down to 42.04s (4.3x)
- ghc-lib: 3m5s down to 40.96s (4.5x)

I've applied `diffoscope` to the generated sdist tarballs with and
without this update to ensure that no unexpected differences are
introduced with this change.

https://github.com/digital-asset/ghc-lib/pull/379 reports a less
dramatic speed up of about a third reduction in build time for `stack
runhaskell CI.hs`. The reason for the discrepancy is that `CI.hs`
performs more steps than just the sdist generation, e.g. checking out
GHC's source tree, or building hadrian. These steps are not included in
the above benchmarks, because they are executed in separate Bazel
actions and can be cached separately.

CHANGELOG_BEGIN
CHANGELOG_END

* Update Cabal files

Co-authored-by: Andreas Herrmann <andreas.herrmann@tweag.io>
2022-06-09 09:15:19 +02:00
..
client_server Change Canton synchronization to use the healthcheck endpoint (#13463) 2022-03-30 16:31:13 +02:00
dev_env_tool Bazelify ghc-lib (#12508) 2022-05-19 10:49:16 +02:00
ghc-lib Speed up ghc-lib(-parser) sdist generation (#14076) 2022-06-09 09:15:19 +02:00
java_testing update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
packaging Darwin M1 Support (#13401) 2022-04-07 17:50:53 +02:00
runfiles update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
scala update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
scalapb Enable java_conversions in scalapb (#12784) 2022-02-07 14:57:57 +00:00
sh Bazelify ghc-lib (#12508) 2022-05-19 10:49:16 +02:00
build_environment.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
BUILD.bazel update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
bundle.bzl Bazelify ghc-lib (#12508) 2022-05-19 10:49:16 +02:00
create_workspace.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
damlc_legacy.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
fat_cc_library.bzl Darwin M1 Support (#13401) 2022-04-07 17:50:53 +02:00
ghc_dwarf.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
googleapis-status-proto.patch Update protobuf docs plugin (#11880) 2021-11-25 20:51:24 +00:00
grpc-bazel-apple.patch Pin rules_apple (#6845) 2020-07-24 09:47:05 +02:00
grpc-bazel-mingw.patch Upgrade gRPC C library (#8693) 2021-02-01 16:11:06 +01:00
grpc-haskell-core-cpp-options.patch Replace Hazel by stack_snapshot (#2743) 2019-11-22 14:24:08 +00:00
grpc-haskell-core-upgrade.patch Upgrade gRPC C library (#8693) 2021-02-01 16:11:06 +01:00
haskell_proto3_suite_deriving_defaults.patch Upgrade to GHC 9.0.2 (#12300) 2022-02-01 11:27:11 +01:00
haskell-c2hs.patch Building c2hs (#355) 2019-04-10 15:12:08 +02:00
haskell-ghc-includes.patch Remove haskell-arm-m1.patch from rules_haskell_patches (#13891) 2022-05-23 16:36:22 +02:00
haskell-ghcide-binary-q.patch Enable assertions in Haskell builds (#6853) 2020-07-24 14:29:25 +00:00
haskell-hie-bios.patch Bump ghc-ghcide (#2936) 2019-09-18 10:29:02 +02:00
haskell-js-dgtable.patch Embed shake data-files via TH (#4584) 2020-02-18 17:53:27 +01:00
haskell-js-flot.patch Embed shake data-files via TH (#4584) 2020-02-18 17:53:27 +01:00
haskell-js-jquery.patch Embed shake data-files via TH (#4584) 2020-02-18 17:53:27 +01:00
haskell-lsp-test-no-reexport.patch Bump ghcide and upgrade to lsp/lsp-types (#10139) 2021-06-30 12:16:12 +00:00
haskell-opt.patch Use more deterministic distdir path in haskell_cabal_* (#11963) 2021-12-02 19:25:26 +00:00
haskell-rts-docs.patch Upgrade to GHC 9.0.2 (#12300) 2022-02-01 11:27:11 +01:00
haskell-shake.patch Embed shake data-files via TH (#4584) 2020-02-18 17:53:27 +01:00
haskell-turtle.patch Upgrade to GHC 9.0.2 (#12300) 2022-02-01 11:27:11 +01:00
haskell-windows-extra-libraries.patch Fix Windows linking issues (#13097) 2022-02-28 08:35:30 +01:00
haskell-zip.patch Bump ghcide and upgrade to lsp/lsp-types (#10139) 2021-06-30 12:16:12 +00:00
haskell.bzl Bazelify ghc-lib (#12508) 2022-05-19 10:49:16 +02:00
hlint.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
java.bzl Revert "Switch to Bazel 5 (#12935)" (#12974) 2022-02-16 18:24:00 +00:00
javadoc_library.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
lsp-types-normalisation.patch Upgrade ghcide (#12579) 2022-01-26 16:00:20 +01:00
oracle.bzl Avoid explicitly listing all Oracle tests in build.yml (#12594) 2022-01-26 11:28:44 +00:00
os_info.bzl Darwin M1 Support (#13401) 2022-04-07 17:50:53 +02:00
pkg.bzl Bazelify ghc-lib (#12508) 2022-05-19 10:49:16 +02:00
pom_file.bzl update copyright headers (#12240) 2022-01-03 16:36:51 +00:00
pom_template.xml update daml.com URLs (#13226) 2022-03-09 18:15:40 +00:00
proto.bzl Enable java_conversions in scalapb (#12784) 2022-02-07 14:57:57 +00:00
remote_apis_no_services.patch Clean broken entries from the Bazel cache (#8668) 2021-01-28 17:57:09 +00:00
rules_nodejs_bash.patch open-sourcing daml 2019-04-04 09:33:38 +01:00
rules_nodejs_node_dependency.patch Upgrade rules-nodejs (#11290) 2021-10-19 14:13:55 +00:00
rules_nodejs_npm_cli_path.patch Upgrade rules-nodejs (#11290) 2021-10-19 14:13:55 +00:00
scala_version.bzl Upgrade to Scala 2.13.8 (#12506) 2022-02-03 09:05:31 +00:00
scala-escape-jvmflags.patch Upgrade rules_scala (#6883) 2020-07-28 08:53:12 +00:00
scala.bzl Drop unused silencer variable (#12640) 2022-01-28 08:17:16 +01:00
scalapb.bzl upgrade scalapb/netty/grpc/protobuf in proven combination (#12628) 2022-02-03 08:38:38 -05:00