Commit Graph

130 Commits

Author SHA1 Message Date
Edward Amsden
8d8dc826cb
docker-image: Pass -t flag to vere if not in an interactive container
Vere will not run a ship if STDIN is closed, unless passed the `-t`
flag. Hosted docker containers will not be interactive, but we would
like the docker image to be usable locally and interactively as well.

Thus, we check for STDIN being closed in the shell prior to launching
Vere, and if it is closed, pass the `-t` flag.
2021-01-16 19:44:04 -05:00
Edward Amsden
d3c5aa74c1
build: A 'smart' docker image
This commit adds the `docker-image` attribute to the main Nix
entrypoint, invoking `nix/pkgs/docker-image` which will build
a 'smart' docker image that can load keyfiles or a pier and
boot a ship

It includes a README for the official docker image, suitable
for posting as the README to a Docker Hub or similar docker
image repository.
2021-01-13 07:37:28 -05:00
Joe Bryan
3e4d495500 nix: remove lib/base64 from solid pill staging 2020-12-08 16:22:58 -08:00
Joe Bryan
5980fbedd0 ci: updates herb call to -test, matching new invocation 2020-12-04 21:46:05 -08:00
Joe Bryan
548440a85d nix: trims trailing whitespace from solid pill builder 2020-11-20 15:23:56 -08:00
Joe Bryan
b1e3b0def2 hoon: moves $xray into userspace 2020-11-20 15:23:56 -08:00
Joe Bryan
aea0f571d7 hoon: moves $plum and +plume to userspace 2020-11-20 15:23:56 -08:00
Brendan Hay
2e72724d44
build: github actions workflow configuration 2020-11-15 15:05:13 +01:00
Brendan Hay
12e1a53633
build: silence service account activation output 2020-11-06 12:00:23 +01:00
Brendan Hay
b7d417a4b2
build: minor refactoring of haskell-nix overlays 2020-11-06 10:22:34 +01:00
Brendan Hay
983261a3d8
build: move darwin install_name_tool fixup from vere to king haskell
As well as some variable naming consistency to appease the nixpkgs
gremlins and keep the git history lords on their toes.
2020-11-06 09:47:11 +01:00
Brendan Hay
767a6f85a4 build: remove {sha256,md5} output for push-storage-object effects 2020-11-05 11:39:05 -08:00
Brendan Hay
e9cf0a3518 build: add log message when destination object already exists 2020-11-05 11:39:04 -08:00
Brendan Hay
244a9b6a06 build: force google-cloud-sdk to use python3
This prevents an error where the python2 version of the SDK tries to
use the monotonic library which is unable to detect the host operating
system inside the sandbox - exploding in typical python fashion.

By using python3 it obviates the need for the monotonic dependency by
using the builtin time.monotonic call.
2020-11-05 11:39:01 -08:00
Brendan Hay
b1790b1b3a build: adding support for hercules ci effects 2020-11-05 11:38:58 -08:00
Brendan Hay
3a0b975f4d
build: expose configurable arguments when booting/testing fake ships
This also sets -l (lite boot) by default for boot-fake-ship to speed
up the initial pier creation used by pills/tests, and uses
urbit.meta.arguments by default for tests - seems a little less likely
to footgun.
2020-11-05 10:25:31 +01:00
Brendan Hay
644acd5002
build: ensure urbit tests are run with the -g argument 2020-11-05 10:25:31 +01:00
Brendan Hay
dd782e062e
vere: ensure debug symbols aren't stripped by default (by nix) 2020-11-05 10:25:30 +01:00
Brendan Hay
8d0a65bb6e
vere: remove dependency on nix-store path for static darwin binary
Poached from @eglaysher's work in #3380.
2020-11-04 20:50:45 +01:00
Brendan Hay
9e1f75da64
build: simplify build tool selection for shell.nix 2020-11-04 15:34:03 +01:00
Brendan Hay
24c63cb078
build: reformatting and triggering tests 2020-11-04 12:55:27 +01:00
Brendan Hay
684005e200
build: disable full test-suite on darwin
Linux remains enabled. This prevents CI errors while figuring out
darwinAllowLocalNetworking and sandboxing.
2020-11-04 12:45:40 +01:00
Brendan Hay
00d236501e
build: fix bind errors in darwin sandboxes
See 5f6840fbb4/src/libstore/build.cc (L2961)
2020-11-04 11:48:46 +01:00
Brendan Hay
cf5c2a40ab
build: remove unnecessary test platform dependence
An alternative approach is being considerd for more granular platform
dependence for tests.
2020-11-04 11:11:39 +01:00
Joe Bryan
52308c4977 test: cleans up output, adds |trim and |meld
with intervening |mass measurements
2020-11-03 01:18:00 -08:00
Brendan Hay
58094c265c
build: bludgeon write/EAGAIN stdout errors into submission 2020-10-30 12:00:56 +01:00
Brendan Hay
6ed18bb825
build: removing shell set options from inline buildPhase for tests 2020-10-29 17:43:23 +01:00
Brendan Hay
6599a85b99
king: ensure tests are enabled on ci and don't depend on lfs pills 2020-10-29 16:20:41 +01:00
Brendan Hay
50f66ce431
build: simplify make test and re-enable on darwin ci for testing 2020-10-29 13:17:05 +01:00
Brendan Hay
fada33dfe6
build: remove checkMaterialization flag 2020-10-28 18:19:53 +01:00
Brendan Hay
59709752eb
build: simplify pushed artefact naming and set object content-type 2020-10-28 12:13:05 +01:00
Brendan Hay
689c0eb6a0
build: ensuring release tarball contains singular parent directory
This ensures parity with the current release tarballs.
2020-10-28 11:05:58 +01:00
Brendan Hay
ce3cbf0857
build: rework all nix expressions to support hercules-ci builds
This also removes nixcrpkgs and OSX cross compilation in favour of
compiling on the target. x86_64/musl targets are still supported
on Linux.

All sources are now managed via niv (see nix/sources.json) and Haskell
package sets are provided/organised via IOHK's haskell.nix.

Some effort has been made to expose similar top-level attributes for
development, but in some cases there have been changes. Please see
the comments in the top-level default.nix and ci.nix files for usage.
2020-10-27 13:55:49 +01:00
botter-nidnul
790e2dd5ec libsigsegv: disable stack vma check on arm
Adds the equivalent arm patches to the i386 patches added in #3054

These result in a dramatic speedup in running  `=a (bex 1.000.000.000)` just as they do on x86_x64

This also sneaks in a hack to `/nix/nixcrpkgs/pkgs/libsigsegv/builder.sh` that allows libsigsegv to configure itself properly when cross compiling release binaries from x86_64 to aarch64.

It won’t trigger for you, since you’re not doing that (yet) but it will make it a little easier for me to maintain my aarch64 static release binaries if that `if [ $host = aarch64-linux-musleabi ]` section is upstreamed.
2020-07-27 23:33:14 -05:00
Philip Monk
e8a307ae67
noun: first pass at memory compaction 2020-07-06 21:35:41 -07:00
Fang
a53e462be8
Merge branch 'master' into ipc-redux 2020-07-03 14:58:49 +02:00
Philip Monk
0574546b77
Merge branch 'ford-fusion' (#3060)
* ford-fusion: (259 commits)
  clay: remove scaffolding
  kiln: make otas continue even if they failed to apply
  metadata-store: add cleanup utility, use on-poke:def instead of no-op
  clay: flop syntax error trace
  landscape/img/codeeval.png: typo in filename
  invite-view: reinstate as potato
  goad: don't crash on pre-OTA sign
  chat-store: responded to comments, cleaned up
  -test: support other desks (full beams in args)
  arvo: use date instead of kelvin
  hoon: re-fix +slab; /tests: fix clay tests
  kiln: don't implicitly create syncs
  gall: don't make large stack trace
  ames: use +cut in +encrypt
  pill: solid
  dojo: print generator type errors
  Squashed commit of the following:
  ci: fix herb tests; update pills
  ford,kiln: nicer errors
  dojo: too many nouns
  ...

Signed-off-by: Philip Monk <phil@pcmonk.me>
2020-06-29 16:07:38 -07:00
Ted Blackman
d7b9889b4a -test: support other desks (full beams in args) 2020-06-28 05:06:16 -04:00
Joe Bryan
14faa22e0f nix: add explicit shutdown to solid-pill derivation 2020-06-26 22:34:20 -07:00
Joe Bryan
6fa194d527 Merge branch 'master' into jb/ipc-redux
* master: (147 commits)
  vere: bump version to 0.10.7
  libsigsegv: disable stack vma check
  vere: bump version to 0.10.6
  ci: add travis as trusted user
  jets: use appropriate macro
  noun: add -C to control memo cache size
  jets: restore fond/play/peek hooks
  jam: add commented-out functionality to count size of atom
  jets: cap memo cache and remove peek, play, and fond jets
  noun: add functions to count size of noun
  release: urbit-os-v1.0.23
  interface/config: fix production build
  soto: run +on-load migration once
  publish, links: restore full height
  sh/build-interface: amend for SPA
  interface/CONTRIBUTING: amend for SPA / webpack
  solid: update pill
  hood + apps: fix OTA process for feat/SPA
  hood: add version %6 for %file-server upgrade
  chat: equally size both code + s3 buttons
  ...
2020-06-26 13:33:00 -07:00
Philip Monk
2713083386
libsigsegv: disable stack vma check
This patches libsigsegv to not check the stack vma on Linux, since that
involves reading procfs, and we make very heavy use of sigsegv.  This
eliminates most of urbit's performance discrepancy between Linux and
MacOS.  These are the benchmarks used; note this is a local MBP vs a
cloud Linux server, and the MBP is almost certainly faster hardware.

We take two benchmarks, one of which decrements 10 million times and the
other simply allocates 125MB of memory.  These are the results:

cpu-heavy ==  =/ n 10.000.000 |-(?~(n n $(n (dec n))))
mem-heavy == =a (bex 1.000.000.008)

macos, cpu-heavy: 6 seconds
macos, mem-heavy: 1 second

linux-before, cpu-heavy: 30 seconds
linux-before, mem-heavy: 160 seconds

linux-after, cpu-heavy 9 seconds
linux-after, mem-heavy 1.3 seconds

This represents a 3x speedup for the cpu-heavy operation and a 120x
speedup for the memory-heavy operation.

This check was used to try to distinguish stack overflow from other
forms of segmentation fault.  In the comments in src/handler-unix.c, it
describes three heuristics it uses, depending on what's available from
the OS.  In the linux-i386 case, all three are availble, so we simply
disable the slow one.  This correctly recognizes stack overflow if you
simply alloca(10000000000).
2020-06-26 11:15:01 -07:00
Ted Blackman
82cfc691ce ci: fix herb tests; update pills 2020-06-25 02:14:16 -04:00
Joe Bryan
dcdd1e4be9 vere: removes now-unused sniproxy dependency 2020-06-23 16:04:40 -07:00
Joe Bryan
829d402dc9 Revert "TMP kill -9 urbit in test"
This reverts commit 6e98bdd3d3.
2020-06-22 14:11:38 -07:00
Joe Bryan
6e4c8361dd build: remove duplicate zlib dependency declaration 2020-06-11 20:31:26 -07:00
Joe Bryan
8fac63661f vere: removes ncurses dependency 2020-06-11 20:25:46 -07:00
Joe Bryan
c3557e0836 Merge branch 'master' into ipc-redux-kh
* master: (25 commits)
  travis: Fix resource exhaustion error in test build.
  travis: Separate job for Haskell on Linux + bump nix version
  gall: properly handle empty outstanding ack queue
  vere: bump version to 0.10.5
  dawn: switch default eth node
  jets: patch cleanup
  Update os1-bug-report.md
  jets: switch openssl to the loom allocator.
  serf: disable automatic |pack
  release: urbit-os-v1.0.20
  behn: fix +unset-timer
  jael: send breach notifications in order
  jael: don't notify breach on initial update to ship
  jael: look for correct wire
  vere: handle failed cue of private key file
  u3: switches GMP import assert to conditional
  u3: fixes rock:load size printf
  vere: bypass terminal logging on lmdb worker thread
  vere: prevents term.c from retrying tcsetattr() indefinitely
  jets: jet decryption of aes-siv mode.
  ...
2020-06-11 19:07:13 -07:00
Benjamin Summers
2501cb0e17 travis: Fix resource exhaustion error in test build. 2020-06-11 14:54:34 -07:00
~siprel
382ace5f23 travis: Various fixes and improvements.
1. Fix bug in test builds that was causing failures for some reason I
   still don't understand.

  `tee` output to stderr was failing with "resource temporarily
  unavailable". The hack that fixed it was to simply write the herb
  output to a file and dump it to stdout after it fully completes.

2. sh/cachix works without CACHIX keys. Simply doesn't upload in that case.

3. Write code to cache testbus builds (disabled for now)

4. fakeship builds get further in bootstrap sequence before committing.

  This fixes problems with PRs from forked repos and enables the scripts
  to be run locally.
2020-06-11 20:34:50 +00:00
~siprel
f853557472 travis: Cache test results in cachix. 2020-06-11 01:00:26 +00:00