Commit Graph

204 Commits

Author SHA1 Message Date
Paul Driver
58e4915a8d stops trying to hide pointer shapes, cleans up secp api, updates secp dependency 2020-09-29 11:39:43 -07:00
Paul Driver
f5449204e1 secp_make->urcrypt 2020-08-29 11:55:58 -07:00
Paul Driver
8d657e12c1 aes-siva-en->urcrypt 2020-08-24 14:38:26 -07:00
Paul Driver
6fdc65dea6 argon2 2020-08-10 15:19:32 -07:00
Paul Driver
cd4b8cb9b0 aes_ecb -> urcrypt 2020-08-05 16:06:04 -07:00
Paul Driver
74d06deb1b one working function in so 2020-07-30 13:07:13 -07:00
botter-nidnul
790e2dd5ec libsigsegv: disable stack vma check on arm
Adds the equivalent arm patches to the i386 patches added in #3054

These result in a dramatic speedup in running  `=a (bex 1.000.000.000)` just as they do on x86_x64

This also sneaks in a hack to `/nix/nixcrpkgs/pkgs/libsigsegv/builder.sh` that allows libsigsegv to configure itself properly when cross compiling release binaries from x86_64 to aarch64.

It won’t trigger for you, since you’re not doing that (yet) but it will make it a little easier for me to maintain my aarch64 static release binaries if that `if [ $host = aarch64-linux-musleabi ]` section is upstreamed.
2020-07-27 23:33:14 -05:00
Paul Driver
b8db4141bc urcrypt pkg with one function, building 2020-07-27 19:19:17 -07:00
Philip Monk
e8a307ae67
noun: first pass at memory compaction 2020-07-06 21:35:41 -07:00
Fang
a53e462be8
Merge branch 'master' into ipc-redux 2020-07-03 14:58:49 +02:00
Philip Monk
0574546b77
Merge branch 'ford-fusion' (#3060)
* ford-fusion: (259 commits)
  clay: remove scaffolding
  kiln: make otas continue even if they failed to apply
  metadata-store: add cleanup utility, use on-poke:def instead of no-op
  clay: flop syntax error trace
  landscape/img/codeeval.png: typo in filename
  invite-view: reinstate as potato
  goad: don't crash on pre-OTA sign
  chat-store: responded to comments, cleaned up
  -test: support other desks (full beams in args)
  arvo: use date instead of kelvin
  hoon: re-fix +slab; /tests: fix clay tests
  kiln: don't implicitly create syncs
  gall: don't make large stack trace
  ames: use +cut in +encrypt
  pill: solid
  dojo: print generator type errors
  Squashed commit of the following:
  ci: fix herb tests; update pills
  ford,kiln: nicer errors
  dojo: too many nouns
  ...

Signed-off-by: Philip Monk <phil@pcmonk.me>
2020-06-29 16:07:38 -07:00
Ted Blackman
d7b9889b4a -test: support other desks (full beams in args) 2020-06-28 05:06:16 -04:00
Joe Bryan
14faa22e0f nix: add explicit shutdown to solid-pill derivation 2020-06-26 22:34:20 -07:00
Joe Bryan
6fa194d527 Merge branch 'master' into jb/ipc-redux
* master: (147 commits)
  vere: bump version to 0.10.7
  libsigsegv: disable stack vma check
  vere: bump version to 0.10.6
  ci: add travis as trusted user
  jets: use appropriate macro
  noun: add -C to control memo cache size
  jets: restore fond/play/peek hooks
  jam: add commented-out functionality to count size of atom
  jets: cap memo cache and remove peek, play, and fond jets
  noun: add functions to count size of noun
  release: urbit-os-v1.0.23
  interface/config: fix production build
  soto: run +on-load migration once
  publish, links: restore full height
  sh/build-interface: amend for SPA
  interface/CONTRIBUTING: amend for SPA / webpack
  solid: update pill
  hood + apps: fix OTA process for feat/SPA
  hood: add version %6 for %file-server upgrade
  chat: equally size both code + s3 buttons
  ...
2020-06-26 13:33:00 -07:00
Philip Monk
2713083386
libsigsegv: disable stack vma check
This patches libsigsegv to not check the stack vma on Linux, since that
involves reading procfs, and we make very heavy use of sigsegv.  This
eliminates most of urbit's performance discrepancy between Linux and
MacOS.  These are the benchmarks used; note this is a local MBP vs a
cloud Linux server, and the MBP is almost certainly faster hardware.

We take two benchmarks, one of which decrements 10 million times and the
other simply allocates 125MB of memory.  These are the results:

cpu-heavy ==  =/ n 10.000.000 |-(?~(n n $(n (dec n))))
mem-heavy == =a (bex 1.000.000.008)

macos, cpu-heavy: 6 seconds
macos, mem-heavy: 1 second

linux-before, cpu-heavy: 30 seconds
linux-before, mem-heavy: 160 seconds

linux-after, cpu-heavy 9 seconds
linux-after, mem-heavy 1.3 seconds

This represents a 3x speedup for the cpu-heavy operation and a 120x
speedup for the memory-heavy operation.

This check was used to try to distinguish stack overflow from other
forms of segmentation fault.  In the comments in src/handler-unix.c, it
describes three heuristics it uses, depending on what's available from
the OS.  In the linux-i386 case, all three are availble, so we simply
disable the slow one.  This correctly recognizes stack overflow if you
simply alloca(10000000000).
2020-06-26 11:15:01 -07:00
Ted Blackman
82cfc691ce ci: fix herb tests; update pills 2020-06-25 02:14:16 -04:00
Joe Bryan
dcdd1e4be9 vere: removes now-unused sniproxy dependency 2020-06-23 16:04:40 -07:00
Joe Bryan
829d402dc9 Revert "TMP kill -9 urbit in test"
This reverts commit 6e98bdd3d3.
2020-06-22 14:11:38 -07:00
Joe Bryan
6e4c8361dd build: remove duplicate zlib dependency declaration 2020-06-11 20:31:26 -07:00
Joe Bryan
8fac63661f vere: removes ncurses dependency 2020-06-11 20:25:46 -07:00
Joe Bryan
c3557e0836 Merge branch 'master' into ipc-redux-kh
* master: (25 commits)
  travis: Fix resource exhaustion error in test build.
  travis: Separate job for Haskell on Linux + bump nix version
  gall: properly handle empty outstanding ack queue
  vere: bump version to 0.10.5
  dawn: switch default eth node
  jets: patch cleanup
  Update os1-bug-report.md
  jets: switch openssl to the loom allocator.
  serf: disable automatic |pack
  release: urbit-os-v1.0.20
  behn: fix +unset-timer
  jael: send breach notifications in order
  jael: don't notify breach on initial update to ship
  jael: look for correct wire
  vere: handle failed cue of private key file
  u3: switches GMP import assert to conditional
  u3: fixes rock:load size printf
  vere: bypass terminal logging on lmdb worker thread
  vere: prevents term.c from retrying tcsetattr() indefinitely
  jets: jet decryption of aes-siv mode.
  ...
2020-06-11 19:07:13 -07:00
Benjamin Summers
2501cb0e17 travis: Fix resource exhaustion error in test build. 2020-06-11 14:54:34 -07:00
~siprel
382ace5f23 travis: Various fixes and improvements.
1. Fix bug in test builds that was causing failures for some reason I
   still don't understand.

  `tee` output to stderr was failing with "resource temporarily
  unavailable". The hack that fixed it was to simply write the herb
  output to a file and dump it to stdout after it fully completes.

2. sh/cachix works without CACHIX keys. Simply doesn't upload in that case.

3. Write code to cache testbus builds (disabled for now)

4. fakeship builds get further in bootstrap sequence before committing.

  This fixes problems with PRs from forked repos and enables the scripts
  to be run locally.
2020-06-11 20:34:50 +00:00
~siprel
f853557472 travis: Cache test results in cachix. 2020-06-11 01:00:26 +00:00
~siprel
26bd5a9e4b king: Got new IPC working, still needs lots of polish. 2020-05-26 23:01:03 +00:00
Elliot Glaysher
cc3a139440 aes_siv: vendors libaes_siv since it isn't in nix.
This vendors libaes_siv so that we can use it to jet
+sivc:aes:crypto.
2020-05-20 13:51:59 -07:00
Joe Bryan
6e98bdd3d3 TMP kill -9 urbit in test 2020-05-01 00:23:59 -07:00
Jared Tobin
c078c16eba
build: remove urbit-large-log-limit derivation
Made unnecessary by 578593d45.
2020-03-01 23:04:22 +04:00
Jared Tobin
ce3c1a0e21
build: better vere label patch
Moves the secondary URBIT_VERSION assignment to a different hunk to
avoid undesirable patching conflicts.
2020-02-24 15:15:56 +04:00
Jared Tobin
05a0a9092a
build: add configurePhase for urbit derivation
46a438f introduced use of the stdenv generic builder instead of an
explicit builder.sh script.  This was reverted in 1daaf041a as it proved
to cause problems when building on some Linux variants.

This fixes the problem encountered on Linux by calling the standard
configure script with bash in an explicit configurePhase.
2020-02-24 14:40:10 +04:00
Jared Tobin
f66d750e1c
build: label large-log-limit vere version
Ensures that large log limit Vere builds are labelled as such, and are
thus easily identifiable.
2020-02-24 14:40:10 +04:00
Jared Tobin
813261dc31
Revert "Revert "Merge branch 'jt/urbit-large-log' (#2211)""
This reverts a revert, commit 1daaf041a7,
in order to fix the problems that led to it being reverted.
2020-02-24 14:37:41 +04:00
Jared Tobin
1daaf041a7
Revert "Merge branch 'jt/urbit-large-log' (#2211)"
This reverts commit 0fc9056074, reversing
changes made to 1e3a1434e0.

The changes made here evidently broke builds on some Linux variants.
2020-01-31 15:26:15 +04:00
Jared Tobin
a6e15c9418
build: add urbit-large-log derivation
Adds a derivation that uses a patched pkg/urbit/vere/lmdb.c that employs
a sixty gigabyte log limit, instead of the default forty.  This is
useful for ships e.g. ~zod that receive a lot of traffic on mainnet.
2020-01-30 13:57:03 +04:00
Jared Tobin
46a438f26e
build: use generic builder in urbit derivation
'installPhase' captures all the important stuff that builder.sh
previously did, and use of the generic builder allows stages of the
build to be more easily overridden.
2020-01-30 11:20:40 +04:00
Brendan Hay
122e01b13d
build: allow baking multiple pills into docker images
By baking (potentially) multiple pills into an image, we can provide
mainnet vs ropsten images. It is still up to the operator to pass
along the pill path(s) to the entrypoint. For example, using Docker:

docker run --tty urbit -B /share/brass.pill -J /share/ivory.pill ...

The main .image attribute still uses only the solid pill, and image-ropsten
has been provided with brass and ivory pills.

Additionally herb has been added to the image for convenience.
2019-12-18 12:02:18 +01:00
Brendan Hay
249c02fe9f
build: trimming unnecessary image configuration 2019-12-16 09:20:28 +01:00
Jared Tobin
9d02b5e88e
Merge branch 'bh/cross-debug' (#2030)
* bh/cross-debug:
  build: add support for releasing cross-compiled debug binaries

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-06 21:19:20 +08:00
Joe Bryan
68a94f420f nix: actually enables debug symbols in release builds 2019-12-05 17:17:33 -08:00
Jared Tobin
2047d6a676
Merge branch 'debug-release' (#2029)
* debug-release:
  nix: enables debug symbols in release builds

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-12-05 16:49:12 +08:00
Brendan Hay
5339a9a826
build: add support for releasing cross-compiled debug binaries
sh/cross can now be invoked as follows:

sh/cross urbit linux64
sh/cross urbit darwin

sh/cross urbit-debug linux64
sh/cross urbit-debug darwin
2019-12-05 09:38:09 +01:00
Joe Bryan
80628e0cac nix: enables debug symbols in release builds 2019-12-04 22:59:53 -08:00
Joe Bryan
13f61ad373 nix: removes unnecessary use of cat in ivory-header build 2019-12-04 22:57:07 -08:00
Philip Monk
0431c3c073
Merge remote-tracking branch 'origin/jam-cue-rock' into rc 2019-12-02 02:08:37 -08:00
Jared Tobin
25fbb5c54a
arvo-ropsten: revert ALEF to AMES [ci skip]
f035955a renamed %alef to %ames, so the Ropsten Arvo derivation should
be updated accordingly.
2019-12-02 16:04:06 +08:00
Philip Monk
4d1457bbaa
Merge remote-tracking branch 'origin/master' into philip/mall-real 2019-11-24 00:01:04 -08:00
Joe Bryan
9c82eadd11 ci: runs |pack after tests 2019-11-22 17:25:18 -08:00
Philip Monk
a5412f01de
Merge branch 'alef-testnet-merge' into philip/mall-real 2019-11-19 13:03:07 -08:00
Jared Tobin
c9c2e33f63 zuse, build: remove default testnet config
Returns the target %zuse contract configuration to mainnet, and also
tweaks the 'arvo-ropsten' build to use %alef instead of %ames.

Also fixes a merge conflict artifact in nix/ops/default.nix.
2019-11-15 19:26:43 +09:00
Brendan Hay
ee856536b4
build: remove unused let binding from image expression 2019-11-12 08:45:02 +01:00
Jared Tobin
999bc1704e
build: give arvo a high priority
0bdced981e introduced the 'arvo-ropsten' derivation.  Attempting to
install both 'arvo' and 'arvo-ropsten' via nix-env will result in a
priority error; this assigns a higher priority to 'arvo' to resolve the
conflict.

Fixes #1912.
2019-11-05 07:10:04 +08:00
Ted Blackman
9a02c64b45 Merge branch 'master' into alef-testnet-merge
Untested, has not been run.
Uses master pills for now.  Will need refreshing.
2019-11-02 23:37:01 -04:00
Joe Bryan
46934cef17 nix: fix ivory-header error conditions (detecting lfs pointers) 2019-11-01 14:47:57 -07:00
Jared Tobin
0bdced981e
build: add Ropsten derivations for arvo and pills
Arvo testnets typically use Ropsten's deploy of the Azimuth contract
instead of mainnet's.  This commit adds an 'arvo-ropsten' derivation for
Arvo as specialised to Ropsten.

In addition to using a different Azimuth address in %zuse, this also
configures %ames and :acme appropriately for running an alternate
network on Ropsten.  %ames has its protocol version incremented, and
:acme uses the Let's Encrypt staging API.

Included here are derivations for creating brass and ivory pills using
arvo-ropsten, to boot.
2019-10-24 10:03:52 +08:00
Jared Tobin
d8f5836876
Merge branch 'image-build' (#1856)
* image-build:
  build: simplify image build interface

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-10-22 10:25:55 +08:00
Brendan Hay
4813d9829b
build: simplify image build interface
This removes the baked in codedump inspection and wrapper scripts
in favour of downstream tooling overriding this as necessary by
using FROM <image> in their respective dockerfile.
2019-10-22 10:18:37 +08:00
Jared Tobin
71b27b0300
Merge branch 'pretty-userspace' (#1847)
* pretty-userspace:
  pills
  hoon: moves new pretty-printer back into userspace

Signed-off-by: Jared Tobin <jared@tlon.io>
2019-10-22 10:12:05 +08:00
Joe Bryan
10645a8d23 build: always include debug symbols 2019-10-17 18:20:23 -07:00
Joe Bryan
fa01bfb0a5 hoon: moves new pretty-printer back into userspace 2019-10-17 17:52:45 -07:00
Jared Tobin
9068188e4b
build: fix 'solid' derivation [ci skip]
578adc93 introduced a 'meta' attribute for handling, amongst other
things, conditional construction of the debug executable.  It missed the
'solid' derivation, causing it to break.
2019-10-15 11:56:16 +04:00
Jared Tobin
8a4a3335d5
build: fix typo [ci skip]
s/ubit/urbit
2019-10-15 11:56:05 +04:00
Ted Blackman
625f855f24 master without pills, hopefully 2019-10-14 16:02:27 -04:00
Brendan Hay
578adc9363
build: move ops debug conditional usage to drv metadata
Rather than conditional reconstructing the "urbit/bin/urbit-debug -g"
string everytime using the debug conditional, this information has been
added to the derivation's .meta attribute.
2019-10-14 10:25:34 +02:00
Brendan Hay
9fee1ffb12
Adding conditional gdb/coredump logging for debug images 2019-10-04 22:31:25 +02:00
Brendan Hay
a312651e58
Adding nix expressions to build/bake docker images 2019-10-03 20:21:41 +02:00
Philip Monk
bd36e61b22
Merge remote-tracking branch 'origin/master' into philip/kale 2019-08-08 14:51:14 -07:00
Philip Monk
900e7fc12c
review fixes 2019-08-07 13:44:41 -07:00
Philip Monk
b0dee239e7
Merge remote-tracking branch 'origin/master' into philip/kale 2019-08-06 15:47:20 -07:00
Joe Bryan
c6a90db71a updates solid pill staging to handle :lens and +solid changes 2019-08-06 11:03:14 -07:00
Joe Bryan
8801854d4c updates :lens app during solid pill staging 2019-08-05 16:34:17 -07:00
Philip Monk
21e4baa2ed
Merge remote-tracking branch 'origin/master' into philip/kale 2019-08-05 13:02:31 -07:00
Joe Bryan
47d7d48928 moves ca-bundle header generation into a separate nix derivation 2019-08-02 13:57:34 -07:00
Joe Bryan
e9def4d3d8 moves ivory header generation into a separate nix derivation 2019-08-02 13:55:15 -07:00
Philip Monk
1391ecafd3
pills and fix pill generation 2019-08-01 13:17:14 -07:00
Joe Bryan
0d3df28fdb automatically generates header for embedded ivory.pill 2019-07-23 12:25:26 -07:00
Logan Allen
d4b5e0114c Added update-ivory-pill script 2019-07-19 16:05:46 -07:00
Joe Bryan
77a23ee542 Merge branch 'v0.8.0.rc' into xcompile-curl-tls
* v0.8.0.rc:
  Move urbit output binary to a build directory.
  80 cols
  changed name of app to be consistent
  updated publish files
2019-07-11 17:05:58 -07:00
Joe Bryan
86590c477c enables TLS in curl cross-compilation derivation 2019-07-11 15:58:52 -07:00
Elliot Glaysher
8e2ddd6e42 Move urbit output binary to a build directory. 2019-07-11 15:44:17 -07:00
Elliot Glaysher
061ce208f0
Merge pull request #1324 from urbit/embed-ssl-certs
Embed the nix SSL certificate file into the resulting binary.
2019-07-11 15:42:17 -07:00
Elliot Glaysher
1dd274fec3 Embed the nix SSL certificate file into the resulting binary.
This writes the SSL certs to a temporary file on startup and then
uses environment variables to control OpenSSL and curl so that
they use them. We have to do this because OSX no longer ships the
normal ca pems, and we statically link with these libraries.
2019-07-11 14:43:18 -07:00
Joe Bryan
778e0518cf adds |mass to CI 2019-07-10 00:33:41 -07:00
Joe Bryan
3ee06f2992 updates lib/pill during solid pill staging 2019-07-03 16:02:49 -07:00
Jared Tobin
2000a14774
Force removal of ./pier/stage directory.
This is required to prevent `rm -r ./pier/stage` from crashing the
script when ./pier/stage doesn't exist, and seems to be necessary for
soliding a pill with the associated derivation.
2019-07-02 15:02:54 +08:00
Joe Bryan
a67a9a75a6 adds ge-additions package to urbit nix-shell config 2019-06-28 11:37:18 -07:00
Joe Bryan
7a25d525fd reset arvo permissions after replacing mount point in ops/solid 2019-06-27 17:32:16 -07:00
Elliot Glaysher
f8c226042e
Merge pull request #1300 from urbit/rings
Ring signature support on Azimuth points
2019-06-24 17:08:04 -07:00
Elliot Glaysher
f1622fd3c2 Minus the stdenv? 2019-06-24 15:53:33 -07:00
Elliot Glaysher
c6bee9593c Maybe we have to manually add CFLAGS like release.sh in urbit? 2019-06-24 15:45:35 -07:00
Elliot Glaysher
abdb1e234b Ben says this might work. 2019-06-24 15:06:49 -07:00
Elliot Glaysher
4eefedabad Actually add the cross.nix file. 2019-06-24 14:55:02 -07:00
Elliot Glaysher
44ddebdb4f Attempt at getting cross-compilation working. 2019-06-24 14:41:54 -07:00
Elliot Glaysher
496e62e01d Merge branch 'cc-release' into rings
This translates the meson build to nix. Previously, we had our ge-additions
package just manually compiled in instead of having its own file.
2019-06-24 13:47:53 -07:00
Joe Bryan
4b005f8144 uses a separate desk and manual commit for +solid staging (avoids -A) 2019-05-15 22:25:19 -07:00
benjamin-tlon
ade1e59ce1
Get cross-compilation ready for release. (#1263)
* Add cross-compilation for `lmdb`.
* Got built caching working in CI with `cachix`.
* Cache cross compilation dependencies and toolchains.
* Do release builds in CI.
* Upload release builds to `bootstrap.urbit.org` on successful build.
* Lots of optimization work for CI.
* Boot from a solid pill in CI and load arvo with `-A`.
* Increase `vere` HTTP timeout to 15m.
2019-05-02 13:13:48 -07:00
Benjamin Summers
8af0f5fa77 Use cachix in CI. 2019-04-25 17:05:16 -07:00
benjamin-tlon
1c4732ca22
Run urbit tests on the debug build, with GC enabled. (#1252)
* Run tests with `urbit-debug` and enable GC while running tests.
* Build `urbit-debug` with `-O0`.
2019-04-25 16:45:39 -07:00
Elliot Glaysher
0a0d7e0e57
Merge pull request #1248 from urbit/cc-lmdb-log-storage
Use lmdb for log storage instead of hand-rolled structure
2019-04-25 12:50:49 -07:00
benjamin-tlon
3c3f7e761e
Misc cleanup blocking CC-Release. (#1249)
* Move extraneous stuff out of pkg/urbit/*
* s/urb/herb/g
* Removed some boilerplate for `urbit` builds.
* Build urbit tests and run them in the nix build.
2019-04-24 17:27:27 -07:00
benjamin-tlon
edd57d380d
Finish cc-release cross-compilation. (#1202)
- Fixes the IPC bug
- Fixes the terminfo bug
- Moves the OSX SDK out of our nixcrpkgs fork.
- Vendor nixcrpkgs instead of having it be a submodule.
2019-04-23 19:50:38 -07:00
Elliot Glaysher
2bef1a30da First minor integration with lmdb.
This stores the ship's identity information in a separate META table in
the lmdb datastore.
2019-04-18 14:01:44 -07:00
Joe Bryan
5aaba00cb0 restores deterministic +test entropy seed 2019-04-02 11:56:16 -07:00
benjamin-tlon
4024cdb739
Fixed Nix build errors on Bernardo's Ubuntu. (#1220) 2019-03-06 11:57:31 -08:00
benjamin-tlon
c86879e833
Nix Build + Monorepo Structure (#1196) 2019-03-04 16:43:53 -08:00