Commit Graph

101 Commits

Author SHA1 Message Date
Brendan Hay
fada33dfe6
build: remove checkMaterialization flag 2020-10-28 18:19:53 +01:00
Brendan Hay
59709752eb
build: simplify pushed artefact naming and set object content-type 2020-10-28 12:13:05 +01:00
Brendan Hay
689c0eb6a0
build: ensuring release tarball contains singular parent directory
This ensures parity with the current release tarballs.
2020-10-28 11:05:58 +01:00
Brendan Hay
ce3cbf0857
build: rework all nix expressions to support hercules-ci builds
This also removes nixcrpkgs and OSX cross compilation in favour of
compiling on the target. x86_64/musl targets are still supported
on Linux.

All sources are now managed via niv (see nix/sources.json) and Haskell
package sets are provided/organised via IOHK's haskell.nix.

Some effort has been made to expose similar top-level attributes for
development, but in some cases there have been changes. Please see
the comments in the top-level default.nix and ci.nix files for usage.
2020-10-27 13:55:49 +01:00
botter-nidnul
790e2dd5ec libsigsegv: disable stack vma check on arm
Adds the equivalent arm patches to the i386 patches added in #3054

These result in a dramatic speedup in running  `=a (bex 1.000.000.000)` just as they do on x86_x64

This also sneaks in a hack to `/nix/nixcrpkgs/pkgs/libsigsegv/builder.sh` that allows libsigsegv to configure itself properly when cross compiling release binaries from x86_64 to aarch64.

It won’t trigger for you, since you’re not doing that (yet) but it will make it a little easier for me to maintain my aarch64 static release binaries if that `if [ $host = aarch64-linux-musleabi ]` section is upstreamed.
2020-07-27 23:33:14 -05:00
Philip Monk
e8a307ae67
noun: first pass at memory compaction 2020-07-06 21:35:41 -07:00
Fang
a53e462be8
Merge branch 'master' into ipc-redux 2020-07-03 14:58:49 +02:00
Philip Monk
0574546b77
Merge branch 'ford-fusion' (#3060)
* ford-fusion: (259 commits)
  clay: remove scaffolding
  kiln: make otas continue even if they failed to apply
  metadata-store: add cleanup utility, use on-poke:def instead of no-op
  clay: flop syntax error trace
  landscape/img/codeeval.png: typo in filename
  invite-view: reinstate as potato
  goad: don't crash on pre-OTA sign
  chat-store: responded to comments, cleaned up
  -test: support other desks (full beams in args)
  arvo: use date instead of kelvin
  hoon: re-fix +slab; /tests: fix clay tests
  kiln: don't implicitly create syncs
  gall: don't make large stack trace
  ames: use +cut in +encrypt
  pill: solid
  dojo: print generator type errors
  Squashed commit of the following:
  ci: fix herb tests; update pills
  ford,kiln: nicer errors
  dojo: too many nouns
  ...

Signed-off-by: Philip Monk <phil@pcmonk.me>
2020-06-29 16:07:38 -07:00
Ted Blackman
d7b9889b4a -test: support other desks (full beams in args) 2020-06-28 05:06:16 -04:00
Joe Bryan
14faa22e0f nix: add explicit shutdown to solid-pill derivation 2020-06-26 22:34:20 -07:00
Joe Bryan
6fa194d527 Merge branch 'master' into jb/ipc-redux
* master: (147 commits)
  vere: bump version to 0.10.7
  libsigsegv: disable stack vma check
  vere: bump version to 0.10.6
  ci: add travis as trusted user
  jets: use appropriate macro
  noun: add -C to control memo cache size
  jets: restore fond/play/peek hooks
  jam: add commented-out functionality to count size of atom
  jets: cap memo cache and remove peek, play, and fond jets
  noun: add functions to count size of noun
  release: urbit-os-v1.0.23
  interface/config: fix production build
  soto: run +on-load migration once
  publish, links: restore full height
  sh/build-interface: amend for SPA
  interface/CONTRIBUTING: amend for SPA / webpack
  solid: update pill
  hood + apps: fix OTA process for feat/SPA
  hood: add version %6 for %file-server upgrade
  chat: equally size both code + s3 buttons
  ...
2020-06-26 13:33:00 -07:00
Philip Monk
2713083386
libsigsegv: disable stack vma check
This patches libsigsegv to not check the stack vma on Linux, since that
involves reading procfs, and we make very heavy use of sigsegv.  This
eliminates most of urbit's performance discrepancy between Linux and
MacOS.  These are the benchmarks used; note this is a local MBP vs a
cloud Linux server, and the MBP is almost certainly faster hardware.

We take two benchmarks, one of which decrements 10 million times and the
other simply allocates 125MB of memory.  These are the results:

cpu-heavy ==  =/ n 10.000.000 |-(?~(n n $(n (dec n))))
mem-heavy == =a (bex 1.000.000.008)

macos, cpu-heavy: 6 seconds
macos, mem-heavy: 1 second

linux-before, cpu-heavy: 30 seconds
linux-before, mem-heavy: 160 seconds

linux-after, cpu-heavy 9 seconds
linux-after, mem-heavy 1.3 seconds

This represents a 3x speedup for the cpu-heavy operation and a 120x
speedup for the memory-heavy operation.

This check was used to try to distinguish stack overflow from other
forms of segmentation fault.  In the comments in src/handler-unix.c, it
describes three heuristics it uses, depending on what's available from
the OS.  In the linux-i386 case, all three are availble, so we simply
disable the slow one.  This correctly recognizes stack overflow if you
simply alloca(10000000000).
2020-06-26 11:15:01 -07:00
Ted Blackman
82cfc691ce ci: fix herb tests; update pills 2020-06-25 02:14:16 -04:00
Joe Bryan
dcdd1e4be9 vere: removes now-unused sniproxy dependency 2020-06-23 16:04:40 -07:00
Joe Bryan
829d402dc9 Revert "TMP kill -9 urbit in test"
This reverts commit 6e98bdd3d3.
2020-06-22 14:11:38 -07:00
Joe Bryan
6e4c8361dd build: remove duplicate zlib dependency declaration 2020-06-11 20:31:26 -07:00
Joe Bryan
8fac63661f vere: removes ncurses dependency 2020-06-11 20:25:46 -07:00
Joe Bryan
c3557e0836 Merge branch 'master' into ipc-redux-kh
* master: (25 commits)
  travis: Fix resource exhaustion error in test build.
  travis: Separate job for Haskell on Linux + bump nix version
  gall: properly handle empty outstanding ack queue
  vere: bump version to 0.10.5
  dawn: switch default eth node
  jets: patch cleanup
  Update os1-bug-report.md
  jets: switch openssl to the loom allocator.
  serf: disable automatic |pack
  release: urbit-os-v1.0.20
  behn: fix +unset-timer
  jael: send breach notifications in order
  jael: don't notify breach on initial update to ship
  jael: look for correct wire
  vere: handle failed cue of private key file
  u3: switches GMP import assert to conditional
  u3: fixes rock:load size printf
  vere: bypass terminal logging on lmdb worker thread
  vere: prevents term.c from retrying tcsetattr() indefinitely
  jets: jet decryption of aes-siv mode.
  ...
2020-06-11 19:07:13 -07:00
Benjamin Summers
2501cb0e17 travis: Fix resource exhaustion error in test build. 2020-06-11 14:54:34 -07:00
~siprel
382ace5f23 travis: Various fixes and improvements.
1. Fix bug in test builds that was causing failures for some reason I
   still don't understand.

  `tee` output to stderr was failing with "resource temporarily
  unavailable". The hack that fixed it was to simply write the herb
  output to a file and dump it to stdout after it fully completes.

2. sh/cachix works without CACHIX keys. Simply doesn't upload in that case.

3. Write code to cache testbus builds (disabled for now)

4. fakeship builds get further in bootstrap sequence before committing.

  This fixes problems with PRs from forked repos and enables the scripts
  to be run locally.
2020-06-11 20:34:50 +00:00
~siprel
f853557472 travis: Cache test results in cachix. 2020-06-11 01:00:26 +00:00
~siprel
26bd5a9e4b king: Got new IPC working, still needs lots of polish. 2020-05-26 23:01:03 +00:00
Elliot Glaysher
cc3a139440 aes_siv: vendors libaes_siv since it isn't in nix.
This vendors libaes_siv so that we can use it to jet
+sivc:aes:crypto.
2020-05-20 13:51:59 -07:00
Joe Bryan
6e98bdd3d3 TMP kill -9 urbit in test 2020-05-01 00:23:59 -07:00
Jared Tobin
c078c16eba
build: remove urbit-large-log-limit derivation
Made unnecessary by 578593d45.
2020-03-01 23:04:22 +04:00
Jared Tobin
ce3c1a0e21
build: better vere label patch
Moves the secondary URBIT_VERSION assignment to a different hunk to
avoid undesirable patching conflicts.
2020-02-24 15:15:56 +04:00
Jared Tobin
05a0a9092a
build: add configurePhase for urbit derivation
46a438f introduced use of the stdenv generic builder instead of an
explicit builder.sh script.  This was reverted in 1daaf041a as it proved
to cause problems when building on some Linux variants.

This fixes the problem encountered on Linux by calling the standard
configure script with bash in an explicit configurePhase.
2020-02-24 14:40:10 +04:00
Jared Tobin
f66d750e1c
build: label large-log-limit vere version
Ensures that large log limit Vere builds are labelled as such, and are
thus easily identifiable.
2020-02-24 14:40:10 +04:00
Jared Tobin
813261dc31
Revert "Revert "Merge branch 'jt/urbit-large-log' (#2211)""
This reverts a revert, commit 1daaf041a7,
in order to fix the problems that led to it being reverted.
2020-02-24 14:37:41 +04:00
Jared Tobin
1daaf041a7
Revert "Merge branch 'jt/urbit-large-log' (#2211)"
This reverts commit 0fc9056074, reversing
changes made to 1e3a1434e0.

The changes made here evidently broke builds on some Linux variants.
2020-01-31 15:26:15 +04:00
Jared Tobin
a6e15c9418
build: add urbit-large-log derivation
Adds a derivation that uses a patched pkg/urbit/vere/lmdb.c that employs
a sixty gigabyte log limit, instead of the default forty.  This is
useful for ships e.g. ~zod that receive a lot of traffic on mainnet.
2020-01-30 13:57:03 +04:00
Jared Tobin
46a438f26e
build: use generic builder in urbit derivation
'installPhase' captures all the important stuff that builder.sh
previously did, and use of the generic builder allows stages of the
build to be more easily overridden.
2020-01-30 11:20:40 +04:00
Brendan Hay
122e01b13d
build: allow baking multiple pills into docker images
By baking (potentially) multiple pills into an image, we can provide
mainnet vs ropsten images. It is still up to the operator to pass
along the pill path(s) to the entrypoint. For example, using Docker:

docker run --tty urbit -B /share/brass.pill -J /share/ivory.pill ...

The main .image attribute still uses only the solid pill, and image-ropsten
has been provided with brass and ivory pills.

Additionally herb has been added to the image for convenience.
2019-12-18 12:02:18 +01:00
Brendan Hay
249c02fe9f
build: trimming unnecessary image configuration 2019-12-16 09:20:28 +01:00
Jared Tobin
9d02b5e88e
Merge branch 'bh/cross-debug' (#2030)
* bh/cross-debug:
  build: add support for releasing cross-compiled debug binaries

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-06 21:19:20 +08:00
Joe Bryan
68a94f420f nix: actually enables debug symbols in release builds 2019-12-05 17:17:33 -08:00
Jared Tobin
2047d6a676
Merge branch 'debug-release' (#2029)
* debug-release:
  nix: enables debug symbols in release builds

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-05 16:49:12 +08:00
Brendan Hay
5339a9a826
build: add support for releasing cross-compiled debug binaries
sh/cross can now be invoked as follows:

sh/cross urbit linux64
sh/cross urbit darwin

sh/cross urbit-debug linux64
sh/cross urbit-debug darwin
2019-12-05 09:38:09 +01:00
Joe Bryan
80628e0cac nix: enables debug symbols in release builds 2019-12-04 22:59:53 -08:00
Joe Bryan
13f61ad373 nix: removes unnecessary use of cat in ivory-header build 2019-12-04 22:57:07 -08:00
Philip Monk
0431c3c073
Merge remote-tracking branch 'origin/jam-cue-rock' into rc 2019-12-02 02:08:37 -08:00
Jared Tobin
25fbb5c54a
arvo-ropsten: revert ALEF to AMES [ci skip]
f035955a renamed %alef to %ames, so the Ropsten Arvo derivation should
be updated accordingly.
2019-12-02 16:04:06 +08:00
Philip Monk
4d1457bbaa
Merge remote-tracking branch 'origin/master' into philip/mall-real 2019-11-24 00:01:04 -08:00
Joe Bryan
9c82eadd11 ci: runs |pack after tests 2019-11-22 17:25:18 -08:00
Philip Monk
a5412f01de
Merge branch 'alef-testnet-merge' into philip/mall-real 2019-11-19 13:03:07 -08:00
Jared Tobin
c9c2e33f63 zuse, build: remove default testnet config
Returns the target %zuse contract configuration to mainnet, and also
tweaks the 'arvo-ropsten' build to use %alef instead of %ames.

Also fixes a merge conflict artifact in nix/ops/default.nix.
2019-11-15 19:26:43 +09:00
Brendan Hay
ee856536b4
build: remove unused let binding from image expression 2019-11-12 08:45:02 +01:00
Jared Tobin
999bc1704e
build: give arvo a high priority
0bdced981e introduced the 'arvo-ropsten' derivation.  Attempting to
install both 'arvo' and 'arvo-ropsten' via nix-env will result in a
priority error; this assigns a higher priority to 'arvo' to resolve the
conflict.

Fixes #1912.
2019-11-05 07:10:04 +08:00
Ted Blackman
9a02c64b45 Merge branch 'master' into alef-testnet-merge
Untested, has not been run.
Uses master pills for now.  Will need refreshing.
2019-11-02 23:37:01 -04:00
Joe Bryan
46934cef17 nix: fix ivory-header error conditions (detecting lfs pointers) 2019-11-01 14:47:57 -07:00