urbit/shrub
1
1
Fork 0
mirror of https://github.com/urbit/shrub.git synced 2024-12-21 01:41:37 +03:00
Code Issues Packages Projects Releases Wiki Activity

Merge tag 'urbit-v1.13' into jb/restage/demand

Browse Source 
urbit-v1.13

Arvo 417K
Vere 1.13

This release makes the loom variably sized, up to a new maximum of 4GB.
The default remains 2GB -- opt for the max with `--loom 32`.
(NB: larger sizes may require adding swap space.)

Release Notes

- supports setting the loom size on startup. sizes are specified in
  exponents of 2, from 1MB (20) to 4GB (32); can differ between processes
  (`--loom` and `--urth-loom`); and can be set for relevant subcommands
  (`meld`, `pack`, `next`, &c).

- improves bit-slice performance (for noun de/serialization and many atom jets)
  by ensuring that implementation inner loops can be vectorized.

- adds the `vile` command, for exporting the keyfile from a ship

- adds the `%xray` hint, for printing the bytecode of a given expression
  (ty ~topfet-parmed!)

- adds the `eval` command, for running hoon (from stdin) without booting a ship
  (ty ~mopfel-winrux!)

- adds a better error message when a ship is already running as root
  (ty ~dinleb-rambep!)

Contributions:

Joe Bryan (48):
      u3: use size_t in u3m_init()
      u3: normalize home-road stack after snapshot restoration
      u3: refactors custom allocator init for libraries
      bump
      vere: make `prep` and `next` commands non-interactive (no tty required)
      Merge pull request #5928 from urbit/jb/upgrade-no-tty
      u3: remove references to u3a_bits/u3a_bytes/words/pages
      u3: makes loom length configurable
      u3: improves image-related error messages in events
      Merge branch 'master' into next/vere
      u3: suppress bail:evil printf
      Merge pull request #5959 from urbit/jb/silence-evil
      ur: optimizes bitstream bytes-writer implementation
      ur: optimizes bitstream bytes-reader implementation
      Revert "test: move _test_lily into noun_tests"
      u3: rewrites u3r_chop(), optimizing bloq sizes < 5
      u3: removes loom byte-length in favor of recalculation
      Merge pull request #5957 from mopfel-winrux/eval
      vere: style clean for eval cmd
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge branch 'jb/hotfix' into jb/fix
      Merge tag 'urbit-v1.12' into jb/fix
      Merge commit 'f94875f9ce^2' into jb/release/vere
      Merge commit '264e63330f^2' into jb/release/vere
      u3: backports u3v_soft_peek()
      vere: adds %vile command for extracting keys
      Merge commit '330db284e8^2' into jb/release/vere
      Merge branch 'master' into jb/release/vere
      test: bump loom to 16MB in jam tests
      u3: restages #6001 for release
      u3: adds loom size to u3m_boot() / u3m_boot_lite()
      vere: adds args to set mars/urth looms by binary exponent
      Merge branch 'jb/restage/loom-set' into jb/release/vere
      vere: supports --loom for all relevant subcommands
      Merge branch 'jb/loom-everywhere' into jb/release/vere
      Merge branch 'jb/release/vere' into jb/restage/chop-fast
      u3: bumps maximum reference bits on all platforms
      vere: preserve 2GB defaults for --loom and --loom-urth
      Merge branch 'jb/restage/chop-fast' into jb/release/vere
      Merge branch 'jb/restage/bump' into jb/release/vere
      Merge branch 'jb/fix' into jb/release/vere
      vere: bumps version
      vere: bumps benchmarks to 16MB loom, initializes guard page
      vere: supports --loom in eval cmd
      vere: bumps version

DaneBettis (22):
      Adds %meme and %xray hints.
      adds xray test, reorders tests, fixes sanity checks
      more notes and debugging for xray
      more xray debugs via ~master-morzod
      more xray debugs thanks to ~master-morzod
      partial fix for indexed bytecode args; more to come
      silenced the broken hoon tests
      cleaner expression and docs
      added macro to clarify _slog_bytecode and cleaned hoon hint tests
      first cleanup from meeting with master-morzod
      this branch PR is now for xray only. meme later.
      minimize the diff
      taming int declarations
      docs, tammed ints, better var names, removed redundant funcs
      fixes suggested by master-morzod, and finished int taming
      make _is_indexed() more obvious and less cute
      remove dev-time modifications to manage.c
      WIP stuff re master-morzod
      more refactoring as suggested by master-morzod
      corrected private function names and block comments
      fixed names of vars to follow cvc_TypeLetter style
      simplified xray callsites and rendering

mopfel-winrux (15):
      Added option to run the eval it checks for and prints whatever you pass it as an arguement
      Added basic nock increment evaulation to eval function
      Got basic hoon command compiling and executing
      Fixed a double boot_lite
      Added vase and stack trace wrapper to input, run hoo in virutalization
      added ability to read from stdin rather than being passed an arguement
      changed up how we are executing hoon. This fixes the issue we see with rpevious versions
      cleaned up code
      merged to master fixed conflict in motes.h
      fixed a few of the issues ~master-morzod pointed out
      changed error messages when booting the ivory pill
      changed how the stdin is concatenated. this version should have less memory leaks
      fixed memory leak and cleaned up style
      removed unneeded checks
      Merge branch 'urbit:master' into eval

pkova (1):
      vere: check for permission error on old process kill
This commit is contained in:
Joe Bryan 2022-11-23 11:24:49 -05:00
commit ea6f239811
9 changed files with 745 additions and 420 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
pkg/urbit/bench/ur_bench.c
View File

@ -7,8 +7,9 @@
static void
_setup(void)
{
u3m_init(1 << 23);
u3m_init(1 << 24);
u3m_pave(c3y);
u3e_init();
}
/* _ames_writ_ex(): |hi packet from fake ~zod to fake ~nec

41
pkg/urbit/daemon/main.c
View File

@ -176,8 +176,8 @@ _main_init(void)
u3_Host.ops_u.hap_w = 50000;
u3_Host.ops_u.kno_w = DefaultKernel;
u3_Host.ops_u.lut_y = u3a_bits + 2;
u3_Host.ops_u.lom_y = u3a_bits + 2;
u3_Host.ops_u.lut_y = u3a_bits + 1;
u3_Host.ops_u.lom_y = u3a_bits + 1;
}
/* _main_pier_run(): get pier from binary path (argv[0]), if appropriate
@ -1227,6 +1227,41 @@ _cw_eval_get_input(FILE* fil_u, size_t siz_i)
static void
_cw_eval(c3_i argc, c3_c* argv[])
{
c3_i ch_i, lid_i;
c3_w arg_w;
static struct option lop_u[] = {
{ "loom", required_argument, NULL, c3__loom },
{ NULL, 0, NULL, 0 }
};
while ( -1 != (ch_i=getopt_long(argc, argv, "", lop_u, &lid_i)) ) {
switch ( ch_i ) {
case c3__loom: {
c3_w lom_w;
c3_o res_o = _main_readw(optarg, u3a_bits + 3, &lom_w);
if ( (c3n == res_o) || (lom_w < 20) ) {
fprintf(stderr, "error: --loom must be >= 20 and <= %u\r\n", u3a_bits + 2);
exit(1);
}
u3_Host.ops_u.lom_y = lom_w;
} break;
case '?': {
fprintf(stderr, "invalid argument\r\n");
exit(1);
} break;
}
}
// argv[optind] is always "eval"
//
if ( optind + 1 != argc ) {
fprintf(stderr, "invalid command\r\n");
exit(1);
}
c3_c* evl_c = _cw_eval_get_input(stdin, 10);
// initialize the Loom and load the Ivory Pill
@ -1238,7 +1273,7 @@ _cw_eval(c3_i argc, c3_c* argv[])
u3_weak pil;
u3C.wag_w |= u3o_hashless;
u3m_boot_lite(u3a_bytes);
u3m_boot_lite((size_t)1 << u3_Host.ops_u.lom_y);
sil_u = u3s_cue_xeno_init_with(ur_fib27, ur_fib28);
if ( u3_none == (pil = u3s_cue_xeno_with(sil_u, len_d, byt_y)) ) {
printf("lite: unable to cue ivory pill\r\n");

8
pkg/urbit/include/c/portable.h
View File

@ -157,24 +157,24 @@
# else
# define U3_OS_LoomBase 0x36000000
# endif
# define U3_OS_LoomBits 29
# define U3_OS_LoomBits 30
# elif defined(U3_OS_mingw)
# define U3_OS_LoomBase 0x28000000000
# define U3_OS_LoomBits 29
# define U3_OS_LoomBits 30
# elif defined(U3_OS_osx)
# ifdef __LP64__
# define U3_OS_LoomBase 0x28000000000
# else
# define U3_OS_LoomBase 0x4000000
# endif
# define U3_OS_LoomBits 29
# define U3_OS_LoomBits 30
# elif defined(U3_OS_bsd)
# ifdef __LP64__
# define U3_OS_LoomBase 0x200000000
# else
# define U3_OS_LoomBase 0x4000000
# endif
# define U3_OS_LoomBits 29
# define U3_OS_LoomBits 30
# else
# error "port: LoomBase"
# endif

34
pkg/urbit/include/noun/retrieve.h
View File

@ -363,11 +363,43 @@
u3r_bytes_all(c3_w* len_w,
u3_atom a);
/* u3r_chop_bits():
**
** XOR `wid_d` bits from`src_w` at `bif_g` to `dst_w` at `bif_g`
**
** NB: [dst_w] must have space for [bit_g + wid_d] bits
*/
void
u3r_chop_bits(c3_g bif_g,
c3_d wid_d,
c3_g bit_g,
c3_w* dst_w,
const c3_w* src_w);
/* u3r_chop_words():
**
** Into the bloq space of `met`, from position `fum` for a
** span of `wid`, to position `tou`, XOR from `src_w`
** into `dst_w`.
**
** NB: [dst_w] must have space for [tou_w + wid_w] bloqs
*/
void
u3r_chop_words(c3_g met_g,
c3_w fum_w,
c3_w wid_w,
c3_w tou_w,
c3_w* dst_w,
c3_w len_w,
const c3_w* src_w);
/* u3r_chop():
**
** Into the bloq space of `met`, from position `fum` for a
** span of `wid`, to position `tou`, XOR from atom `src`
** into ray `dst`.
** into `dst_w`.
**
** NB: [dst_w] must have space for [tou_w + wid_w] bloqs
*/
void
u3r_chop(c3_g met_g,

218
pkg/urbit/noun/retrieve.c
View File

@ -1453,11 +1453,174 @@ u3r_safe_chub(u3_noun dat, c3_d* out_d)
return c3y;
}
/* u3r_chop_bits():
**
** XOR `wid_d` bits from`src_w` at `bif_g` to `dst_w` at `bif_g`
**
** NB: [dst_w] must have space for [bit_g + wid_d] bits
*/
void
u3r_chop_bits(c3_g bif_g,
c3_d wid_d,
c3_g bit_g,
c3_w* dst_w,
const c3_w* src_w)
{
c3_y fib_y = 32 - bif_g;
c3_y tib_y = 32 - bit_g;
// we need to chop words
//
if ( wid_d >= tib_y ) {
// align *dst_w
//
if ( bit_g ) {
c3_w low_w = src_w[0] >> bif_g;
if ( bif_g > bit_g ) {
low_w ^= src_w[1] << fib_y;
}
*dst_w++ ^= low_w << bit_g;
wid_d -= tib_y;
bif_g += tib_y;
src_w += !!(bif_g >> 5);
bif_g &= 31;
fib_y = 32 - bif_g;
}
{
size_t i_i, byt_i = wid_d >> 5;
if ( !bif_g ) {
for ( i_i = 0; i_i < byt_i; i_i++ ) {
dst_w[i_i] ^= src_w[i_i];
}
}
else {
for ( i_i = 0; i_i < byt_i; i_i++ ) {
dst_w[i_i] ^= (src_w[i_i] >> bif_g) ^ (src_w[i_i + 1] << fib_y);
}
}
src_w += byt_i;
dst_w += byt_i;
wid_d &= 31;
bit_g = 0;
}
}
// we need to chop (more) bits
//
if ( wid_d ) {
c3_w hig_w = src_w[0] >> bif_g;
if ( wid_d > fib_y ) {
hig_w ^= src_w[1] << fib_y;
}
*dst_w ^= (hig_w & ((1 << wid_d) - 1)) << bit_g;
}
}
/* u3r_chop_words():
**
** Into the bloq space of `met`, from position `fum` for a
** span of `wid`, to position `tou`, XOR from `src_w`
** into `dst_w`.
**
** NB: [dst_w] must have space for [tou_w + wid_w] bloqs
*/
void
u3r_chop_words(c3_g met_g,
c3_w fum_w,
c3_w wid_w,
c3_w tou_w,
c3_w* dst_w,
c3_w len_w,
const c3_w* src_w)
{
// operate on words
//
if ( met_g >= 5 ) {
size_t i_i, wid_i;
{
c3_g hut_g = met_g - 5;
size_t fum_i = (size_t)fum_w << hut_g;
size_t tou_i = (size_t)tou_w << hut_g;
size_t tot_i;
wid_i = (size_t)wid_w << hut_g;
tot_i = fum_i + wid_i;
// since [dst_w] must have space for (tou_w + wid_w) bloqs,
// neither conversion can overflow
//
if ( (fum_i >> hut_g != fum_w) || (tot_i - wid_i != fum_i) ) {
u3m_bail(c3__fail);
return;
}
else if ( fum_i >= len_w ) {
return;
}
if ( tot_i > len_w ) {
wid_i -= tot_i - len_w;
}
src_w += fum_i;
dst_w += tou_i;
}
for ( i_i = 0; i_i < wid_i; i_i++ ) {
dst_w[i_i] ^= src_w[i_i];
}
}
// operate on bits
//
else {
c3_d wid_d = (c3_d)wid_w << met_g;
c3_g bif_g, bit_g;
{
c3_d len_d = (c3_d)len_w << 5;
c3_d fum_d = (c3_d)fum_w << met_g;
c3_d tou_d = (c3_d)tou_w << met_g;
c3_d tot_d = fum_d + wid_d;
// see above
//
if ( (fum_d >> met_g != fum_w) || (tot_d - wid_d != fum_d) ) {
u3m_bail(c3__fail);
return;
}
else if ( fum_d > len_d ) {
return;
}
if ( tot_d > len_d ) {
wid_d -= tot_d - len_d;
}
src_w += fum_d >> 5;
dst_w += tou_d >> 5;
bif_g = fum_d & 31;
bit_g = tou_d & 31;
}
u3r_chop_bits(bif_g, wid_d, bit_g, dst_w, src_w);
}
}
/* u3r_chop():
**
** Into the bloq space of `met`, from position `fum` for a
** span of `wid`, to position `tou`, XOR from atom `src`
** into `dst_w`.
**
** NB: [dst_w] must have space for [tou_w + wid_w] bloqs
*/
void
u3r_chop(c3_g met_g,
@ -1467,65 +1630,24 @@ u3r_chop(c3_g met_g,
c3_w* dst_w,
u3_atom src)
{
c3_w i_w;
c3_w* src_w;
c3_w len_w;
c3_w* buf_w;
c3_assert(u3_none != src);
c3_assert(_(u3a_is_atom(src)));
if ( _(u3a_is_cat(src)) ) {
len_w = src ? 1 : 0;
buf_w = &src;
src_w = &src;
}
else {
u3a_atom* src_u = u3a_to_ptr(src);
c3_assert(u3_none != src);
c3_assert(_(u3a_is_atom(src)));
len_w = src_u->len_w;
buf_w = src_u->buf_w;
src_w = src_u->buf_w;
}
if ( met_g < 5 ) {
c3_w san_w = (1 << met_g);
c3_w mek_w = ((1 << san_w) - 1);
c3_w baf_w = (fum_w << met_g);
c3_w bat_w = (tou_w << met_g);
// XX: efficiency: poor. Iterate by words.
//
for ( i_w = 0; i_w < wid_w; i_w++ ) {
c3_w waf_w = (baf_w >> 5);
c3_g raf_g = (baf_w & 31);
c3_w wat_w = (bat_w >> 5);
c3_g rat_g = (bat_w & 31);
c3_w hop_w;
hop_w = (waf_w >= len_w) ? 0 : buf_w[waf_w];
hop_w = (hop_w >> raf_g) & mek_w;
dst_w[wat_w] ^= (hop_w << rat_g);
baf_w += san_w;
bat_w += san_w;
}
}
else {
c3_g hut_g = (met_g - 5);
c3_w san_w = (1 << hut_g);
c3_w j_w;
for ( i_w = 0; i_w < wid_w; i_w++ ) {
c3_w wuf_w = (fum_w + i_w) << hut_g;
c3_w wut_w = (tou_w + i_w) << hut_g;
for ( j_w = 0; j_w < san_w; j_w++ ) {
dst_w[wut_w + j_w] ^=
((wuf_w + j_w) >= len_w)
? 0
: buf_w[wuf_w + j_w];
}
}
}
u3r_chop_words(met_g, fum_w, wid_w, tou_w, dst_w, len_w, src_w);
}
/* u3r_string(): `a` as malloced C string.

68
pkg/urbit/tests/boot_tests.c Normal file
View File

@ -0,0 +1,68 @@
#include "all.h"
#include "ur/ur.h"
#include "vere/ivory.h"
#include "vere/vere.h"
/* _setup(): prepare for tests.
*/
static void
_setup(void)
{
c3_d len_d = u3_Ivory_pill_len;
c3_y* byt_y = u3_Ivory_pill;
u3_cue_xeno* sil_u;
u3_weak pil;
u3C.wag_w |= u3o_hashless;
u3m_boot_lite(1 << 26);
sil_u = u3s_cue_xeno_init_with(ur_fib27, ur_fib28);
if ( u3_none == (pil = u3s_cue_xeno_with(sil_u, len_d, byt_y)) ) {
printf("*** fail _setup 1\n");
exit(1);
}
u3s_cue_xeno_done(sil_u);
if ( c3n == u3v_boot_lite(pil) ) {
printf("*** fail _setup 2\n");
exit(1);
}
}
/* _test_lily(): test small noun parsing.
*/
static void
_test_lily()
{
c3_l lit_l;
c3_w big_w[] = {0, 0, 1};
u3_noun big = u3i_words(3, big_w);
u3_noun cod = u3dc("scot", c3__uv, big);
if ( c3y == u3v_lily(c3__uv, cod, &lit_l) ) {
printf("*** fail _test_lily-1\n");
exit(1);
}
cod = u3dc("scot", c3__ud, 0x7fffffff);
if ( (c3n == u3v_lily(c3__ud, cod, &lit_l)) ||
(0x7fffffff != lit_l) ) {
printf("*** fail _test_lily-2a\n");
exit(1);
}
cod = u3dc("scot", c3__ux, u3i_word(0x80000000));
if ( c3y == u3v_lily(c3__ux, cod, &lit_l) ) {
printf("*** fail _test_lily-2b\n");
exit(1);
}
}
/* main(): run all test cases.
*/
int
main(int argc, char* argv[])
{
_setup();
_test_lily();
fprintf(stderr, "test boot: ok\n");
return 0;
}

660
pkg/urbit/tests/noun_tests.c
View File

@ -1,7 +1,4 @@
#include "all.h"
#include "ur/ur.h"
#include "vere/ivory.h"
#include "vere/vere.h"
#define TRUE 1
#define FALSE 0
@ -11,23 +8,358 @@
static void
_setup(void)
{
c3_d len_d = u3_Ivory_pill_len;
c3_y* byt_y = u3_Ivory_pill;
u3_cue_xeno* sil_u;
u3_weak pil;
u3m_init(1 << 20);
u3m_pave(c3y);
}
u3C.wag_w |= u3o_hashless;
u3m_boot_lite(u3a_bytes);
sil_u = u3s_cue_xeno_init_with(ur_fib27, ur_fib28);
if ( u3_none == (pil = u3s_cue_xeno_with(sil_u, len_d, byt_y)) ) {
printf("*** fail _setup 1\n");
exit(1);
/* _test_u3r_chop: "extract bit slices from atom"
*/
static c3_i
_test_u3r_chop()
{
c3_i ret_i = 1;
c3_w dst_w = 0;
u3_atom src = 0b11011;
// bloq 0
//
{
// read 1 bit from pos=0 (far right)
//
dst_w = 0;
u3r_chop(0, 0, 1, 0, &dst_w, src);
if ( 0x1 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 0, 0\r\n");
ret_i = 0;
}
u3s_cue_xeno_done(sil_u);
if ( c3n == u3v_boot_lite(pil) ) {
printf("*** fail _setup 2\n");
exit(1);
// read 1 bit from pos=1
//
dst_w = 0;
u3r_chop(0, 1, 1, 0, &dst_w, src);
if ( 0x1 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 0, 1\r\n");
ret_i = 0;
}
// read 1 bit from pos=2
//
dst_w = 0;
u3r_chop(0, 2, 1, 0, &dst_w, src);
if ( 0x0 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 0, 2\r\n");
ret_i = 0;
}
// read 4 x 1 bit bloq from pos=0
//
dst_w = 0;
u3r_chop(0, 0, 4, 0, &dst_w, src);
if ( 0b1011 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 0, 3\r\n");
ret_i = 0;
}
// read 4 x 1 bit bloq from pos=0 into offset 1
//
dst_w = 0;
u3r_chop(0, 0, 4, 1, &dst_w, src);
if ( 0b10110 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 0, 4\r\n");
ret_i = 0;
}
}
// bloq 1
//
{
// read 2 bit from pos=0 (far right)
//
dst_w = 0;
u3r_chop(1, 0, 1, 0, &dst_w, src);
if ( 0b11 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 1, 0\r\n");
ret_i = 0;
}
// read 2 bit from pos=1
//
dst_w = 0;
u3r_chop(1, 1, 1, 0, &dst_w, src);
if ( 0b10 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 1, 1\r\n");
ret_i = 0;
}
// read 2 bit from pos=2 (2 bloq over)
dst_w = 0;
u3r_chop(1, 2, 1, 0, &dst_w, src);
if ( 0b01 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 1, 2\r\n");
ret_i = 0;
}
}
// bloq 3
{
dst_w = 0;
u3r_chop(3, 0, 1, 0, &dst_w, src);
if ( 0b11011 != dst_w ) {
fprintf(stderr, "test: u3r_chop: bloq 3, 0\r\n");
ret_i = 0;
}
}
// read 1,8,16 bit bloqs from an indirect atom
//
{
src = u3i_string("abcdefghij");
// 1 bit pos=0 (far right)
//
dst_w = 0;
u3r_chop(0, 0, 1, 0, &dst_w, src);
if ( 0b1 != dst_w ) {
fprintf(stderr, "test: u3r_chop: indirect 0\r\n");
ret_i = 0;
}
// 8 bits pos=0
//
dst_w = 0;
u3r_chop(0, 0, 8, 0, &dst_w, src);
if ( 0b1100001 != dst_w ) {
fprintf(stderr, "test: u3r_chop: indirect 1\r\n");
ret_i = 0;
}
// 1 byte pos=0
//
dst_w = 0;
u3r_chop(3, 0, 1, 0, &dst_w, src);
if ( 0b1100001 != dst_w ) {
fprintf(stderr, "test: u3r_chop: indirect 2\r\n");
ret_i = 0;
}
// 1 short pos=0
//
dst_w = 0;
u3r_chop(4, 0, 1, 0, &dst_w, src);
if ( 0b0110001001100001 != dst_w ) {
fprintf(stderr, "test: u3r_chop: indirect 3\r\n");
ret_i = 0;
}
u3z(src);
}
// read lots of bits from a direct noun which holds 64 bits of data
// makes sure that we handle top 32 / bottom 32 correctly
{
c3_y inp_y[8] = { 0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7 };
src = u3i_bytes(8, inp_y);
c3_w dst_w[2] = {0};
u3r_chop(0, 0, 63, 0, dst_w, src);
if ( (0x3020100 != dst_w[0]) || (0x7060504 != dst_w[1]) ) {
fprintf(stderr, "test: u3r_chop: indirect 4\r\n");
ret_i = 0;
}
u3z(src);
}
// as above (read lots of bits from a direct noun which holds 64 bits of data
// makes sure that we handle top 32 / bottom 32 correctly)
// but with a bit more nuance
{
c3_y inp_y[8] = { 0x0, 0x0, 0x0, 0xaa, 0xff, 0x0, 0x0, 0x0 };
src = u3i_bytes(8, (c3_y*)inp_y);
dst_w = 0;
u3r_chop(0, 24, 16, 0, &dst_w, src);
if ( 0b1111111110101010 != dst_w ) {
fprintf(stderr, "test: u3r_chop: indirect 5\r\n");
ret_i = 0;
}
u3z(src);
}
return ret_i;
}
/* _test_chop_slow(): "golden master" for chop tests (formerly u3r_chop())
*/
void
_test_chop_slow(c3_g met_g,
c3_w fum_w,
c3_w wid_w,
c3_w tou_w,
c3_w* dst_w,
c3_w len_w,
c3_w* buf_w)
{
c3_w i_w;
if ( met_g < 5 ) {
c3_w san_w = (1 << met_g);
c3_w mek_w = ((1 << san_w) - 1);
c3_w baf_w = (fum_w << met_g);
c3_w bat_w = (tou_w << met_g);
// XX: efficiency: poor. Iterate by words.
//
for ( i_w = 0; i_w < wid_w; i_w++ ) {
c3_w waf_w = (baf_w >> 5);
c3_g raf_g = (baf_w & 31);
c3_w wat_w = (bat_w >> 5);
c3_g rat_g = (bat_w & 31);
c3_w hop_w;
hop_w = (waf_w >= len_w) ? 0 : buf_w[waf_w];
hop_w = (hop_w >> raf_g) & mek_w;
dst_w[wat_w] ^= (hop_w << rat_g);
baf_w += san_w;
bat_w += san_w;
}
}
else {
c3_g hut_g = (met_g - 5);
c3_w san_w = (1 << hut_g);
c3_w j_w;
for ( i_w = 0; i_w < wid_w; i_w++ ) {
c3_w wuf_w = (fum_w + i_w) << hut_g;
c3_w wut_w = (tou_w + i_w) << hut_g;
for ( j_w = 0; j_w < san_w; j_w++ ) {
dst_w[wut_w + j_w] ^=
((wuf_w + j_w) >= len_w)
? 0
: buf_w[wuf_w + j_w];
}
}
}
}
/* _test_chop_smol(): test permuations of chop from bloq 0-4
*/
static c3_i
_test_chop_smol(c3_c* cap_c, c3_y val_y)
{
c3_i ret_i = 1;
c3_g met_g;
c3_w fum_w, wid_w, tou_w;
c3_w len_w = 34; // (rsh [0 5] (mul 2 (mul 34 (bex 4))))
c3_w src_w[len_w];
c3_w a_w[len_w];
c3_w b_w[len_w];
memset(src_w, val_y, len_w << 2);
for ( met_g = 0; met_g < 5; met_g++ ) {
for ( fum_w = 0; fum_w <= len_w; fum_w++ ) {
for ( wid_w = 0; wid_w <= len_w; wid_w++ ) {
for ( tou_w = 0; tou_w <= len_w; tou_w++ ) {
memset(a_w, 0, len_w << 2);
memset(b_w, 0, len_w << 2);
u3r_chop_words(met_g, fum_w, wid_w, tou_w, a_w, len_w, src_w);
_test_chop_slow(met_g, fum_w, wid_w, tou_w, b_w, len_w, src_w);
if ( 0 != memcmp(a_w, b_w, len_w << 2) ) {
c3_g sif_g = 5 - met_g;
c3_w mas_w = (1 << met_g) - 1;
c3_w out_w = tou_w >> sif_g;
c3_w max_w = out_w + !!(fum_w & mas_w)
+ (wid_w >> sif_g) + !!(wid_w & mas_w);
fprintf(stderr, "%s (0x%x): met_g=%u fum_w=%u wid_w=%u tou_w=%u\r\n",
cap_c, val_y,
met_g, fum_w, wid_w, tou_w);
fprintf(stderr, "%u-%u: ", out_w, max_w - 1);
for ( ; out_w < max_w; out_w++ ) {
fprintf(stderr, "[0x%x 0x%x] ", a_w[out_w], b_w[out_w]);
}
fprintf(stderr, "\r\n");
}
}
}
}
}
return ret_i;
}
/* _test_chop_huge(): test permuations of chop from bloq 5+
*/
static c3_i
_test_chop_huge(c3_c* cap_c, c3_y val_y)
{
c3_i ret_i = 1;
c3_g met_g;
c3_w fum_w, wid_w, tou_w;
c3_w len_w = 192; // (rsh [0 5] (mul 2 (mul 3 (bex 10))))
c3_w src_w[len_w];
c3_w a_w[len_w];
c3_w b_w[len_w];
memset(src_w, val_y, len_w << 2);
for ( met_g = 5; met_g <= 10; met_g++ ) {
for ( fum_w = 0; fum_w <= 3; fum_w++ ) {
for ( wid_w = 0; wid_w <= 2; wid_w++ ) {
for ( tou_w = 0; tou_w <= 1; tou_w++ ) {
memset(a_w, 0, len_w << 2);
memset(b_w, 0, len_w << 2);
u3r_chop_words(met_g, fum_w, wid_w, tou_w, a_w, len_w, src_w);
_test_chop_slow(met_g, fum_w, wid_w, tou_w, b_w, len_w, src_w);
if ( 0 != memcmp(a_w, b_w, len_w << 2) ) {
c3_g sif_g = met_g - 5;
c3_w mas_w = (1 << met_g) - 1;
c3_w out_w = tou_w << sif_g;
c3_w max_w = out_w + !!(fum_w & mas_w)
+ (wid_w << sif_g) + !!(wid_w & mas_w);
fprintf(stderr, "%s (0x%x): met_g=%u fum_w=%u wid_w=%u tou_w=%u\r\n",
cap_c, val_y,
met_g, fum_w, wid_w, tou_w);
fprintf(stderr, "%u-%u: ", out_w, max_w - 1);
for ( ; out_w < max_w; out_w++ ) {
fprintf(stderr, "[0x%x 0x%x] ", a_w[out_w], b_w[out_w]);
}
fprintf(stderr, "\r\n");
}
}
}
}
}
return ret_i;
}
/* _test_u3r_chop(): bit slice XOR
*/
static c3_i
_test_chop()
{
return _test_u3r_chop()
& _test_chop_smol("chop smol zeros", 0x0)
& _test_chop_smol("chop smol ones", 0xff)
& _test_chop_smol("chop smol alt 1", 0xaa)
& _test_chop_smol("chop smol alt 2", 0x55)
& _test_chop_huge("chop huge zeros", 0x0)
& _test_chop_huge("chop huge ones", 0xff)
& _test_chop_huge("chop huge alt 1", 0xaa)
& _test_chop_huge("chop huge alt 2", 0x55);
}
/* _util_rand_string(): dynamically allocated len_w random string
@ -1265,256 +1597,6 @@ _test_u3r_at()
if (bignum != ret) { printf("*** u3r_at \n"); }
}
/* _test_u3r_chop: "extract bit slices from atom"
*/
static void
_test_u3r_chop()
{
c3_w dst_w = 0;
// read 1 bit bloq
{
// read 1 bit from pos=0 (far right)
u3_atom src = 0b11011;
c3_g bloqsize_g = 0;
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0x1 != dst_w) { printf("*** test_u3r_chop \n"); }
// read 1 bit from pos=1
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
1, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0x1 != dst_w) { printf("*** test_u3r_chop 2\n"); }
// read 1 bit from pos=2
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
2, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0x0 != dst_w) { printf("*** test_u3r_chop 3\n"); }
// read 4 x 1 bit bloq from pos=0
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
4, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b1011 != dst_w) { printf("*** test_u3r_chop 4\n"); }
// read 1 x 1 bit bloq from pos=0 into offset 1
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
4, // count of bloqs
1, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b10110 != dst_w) { printf("*** test_u3r_chop 5\n"); }
}
// read 2 bit bloq
{
u3_atom src = 0b11011;
c3_g bloqsize_g = 1;
// read 2 bit from pos=0 (far right)
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b11 != dst_w) { printf("*** test_u3r_chop 2.1\n"); }
// read 2 bit from pos=1 (1 bloq over )
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
1, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b10 != dst_w) { printf("*** test_u3r_chop 2.2\n"); }
// read 2 bit from pos=2 (2 bloq over)
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
2, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b01 != dst_w) { printf("*** test_u3r_chop 2.3\n"); }
}
// read 8 bit bloq
{
u3_atom src = 0b11011;
c3_g bloqsize_g = 3; // 2^3 = 8 bits
// pos=0 (far right)
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b11011 != dst_w) { printf("*** test_u3r_chop 8.1\n"); }
}
// read 1,8,16 bit bloqs from an indirect atom
{
// build an indirect noun 'src'
c3_c* input_c = "abcdefghij";
u3_noun src = u3i_bytes(10, (c3_y*)input_c);
c3_g bloqsize_g = 0; // 2^0 = 1 bit
// 1 x 1 bit pos=0 (far right)
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b1 != dst_w) {
printf("*** test_u3r_chop indirect.1\n");
}
// 8 x 1 bit pos=0 (far right)
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
8, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b1100001 != dst_w) {
printf("*** test_u3r_chop indirect.2\n");
}
// 1 x 1 byte = 8 bit, pos=0 (far right)
bloqsize_g = 3; // 2^3 = 1 byte
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b1100001 != dst_w) {
printf("*** test_u3r_chop indirect.3\n");
}
// 1 x 16 bit bloq, pos = 0
bloqsize_g = 4; // 2^4 = 2 bytes
dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
0, // start index
1, // count of bloqs
0, // end index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b0110001001100001 != dst_w) {
printf("*** test_u3r_chop indirect.4\n");
}
}
// read lots of bits from a direct noun which holds 64 bits of data
// makes sure that we handle top 32 / bottom 32 correctly
{
// build an indirect noun 'src'
c3_c input_c[8] = { 0x0, 0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7 };
u3_noun src = u3i_bytes(8, (c3_y*)input_c);
c3_g bloqsize_g = 0; // 2^0 = 1 bit
c3_w dst_w[2];
memset(dst_w, 0, 2 * sizeof(c3_w));
u3r_chop(bloqsize_g, /// bloq size
0, // start index
63, // count of bloqs
0, // offset on out index
dst_w, // where bytes go to
src); // where bytes come from
}
// as above (read lots of bits from a direct noun which holds 64 bits of data
// makes sure that we handle top 32 / bottom 32 correctly)
// but with a bit more nuance
{
// least significant most
c3_c input_c[8] = { 0x0, 0x0, 0x0, 0xaa, 0xff, 0x0, 0x0, 0x0 };
u3_noun src = u3i_bytes(8, (c3_y*)input_c);
c3_g bloqsize_g = 0; // 2^0 = 1 bit
c3_w dst_w = 0;
u3r_chop(bloqsize_g, /// bloq size
24, // start index
16, // count of bloqs
0, // offset on out index
& dst_w, // where bytes go to
src); // where bytes come from
if (0b1111111110101010 != dst_w) {
printf("*** test_u3r_chop indirect. 6\n");
}
}
}
// XX disabled, static functions
//
#if 0
@ -1652,31 +1734,17 @@ _test_nvm_stack()
#endif
}
/* _test_lily(): test small noun parsing.
*/
static void
_test_lily()
static c3_i
_test_noun(void)
{
c3_l lit_l;
c3_w big_w[] = {0, 0, 1};
u3_noun big = u3i_words(3, big_w);
u3_noun cod = u3dc("scot", c3__uv, big);
c3_i ret_i = 1;
if ( c3y == u3v_lily(c3__uv, cod, &lit_l) ) {
printf("*** fail _test_lily-1\n");
exit(1);
}
cod = u3dc("scot", c3__ud, 0x7fffffff);
if ( (c3n == u3v_lily(c3__ud, cod, &lit_l)) ||
(0x7fffffff != lit_l) ) {
printf("*** fail _test_lily-2a\n");
exit(1);
}
cod = u3dc("scot", c3__ux, u3i_word(0x80000000));
if ( c3y == u3v_lily(c3__ux, cod, &lit_l) ) {
printf("*** fail _test_lily-2b\n");
exit(1);
if ( !_test_chop() ) {
fprintf(stderr, "test noun: chop failed\r\n");
ret_i = 0;
}
return ret_i;
}
/* main(): run all test cases.
@ -1686,9 +1754,20 @@ main(int argc, char* argv[])
{
_setup();
if ( !_test_noun() ) {
fprintf(stderr, "test noun: failed\r\n");
exit(1);
}
// GC
//
u3m_grab(u3_none);
// XX the following tests leak memory
// fix and move to _test_noun()
//
_test_noun_bits_set();
_test_noun_bits_read();
_test_u3r_chop();
_test_imprison();
_test_imprison_complex();
_test_sing();
@ -1698,7 +1777,6 @@ main(int argc, char* argv[])
_test_cells_complex();
_test_u3r_at();
_test_nvm_stack();
_test_lily();
fprintf(stderr, "test_noun: ok\n");

101
pkg/urbit/ur/bitstream.c
View File

@ -411,25 +411,21 @@ ur_bsr_bytes_any(ur_bsr_t *bsr, uint64_t len, uint8_t *out)
//
else {
uint8_t rest = 8 - off;
uint8_t mask = (1 << off) - 1;
uint8_t byt, l, m = *b >> off;
uint64_t last = left - 1;
uint64_t max = ur_min(last, len_byt);
uint8_t m, l;
// loop over all the bytes we need (or all that remain)
//
// [l] holds [off] bits
// [m] holds [rest] bits
//
{
uint64_t max = ur_min(last, len_byt);
uint64_t i;
for ( i = 0; i < max; i++ ) {
byt = *++b;
l = byt & mask;
out[i] = m ^ (l << rest);
m = byt >> off;
out[i] = (b[i] >> off) ^ (b[i + 1] << rest);
}
b += max;
m = *b >> off;
}
// we're reading into or beyond the last byte [bsr]
@ -441,13 +437,13 @@ ur_bsr_bytes_any(ur_bsr_t *bsr, uint64_t len, uint8_t *out)
uint8_t bits = len - (last << 3);
if ( bits < rest ) {
out[last] = m & ((1 << bits) - 1);
out[max] = m & ((1 << len_bit) - 1);
bsr->bytes = b;
left = 1;
off += len_bit;
}
else {
out[last] = m;
out[max] = m;
bsr->bytes = 0;
left = 0;
off = 0;
@ -465,11 +461,11 @@ ur_bsr_bytes_any(ur_bsr_t *bsr, uint64_t len, uint8_t *out)
if ( len_bit ) {
if ( len_bit <= rest ) {
out[len_byt] = m & ((1 << len_bit) - 1);
out[max] = m & ((1 << len_bit) - 1);
}
else {
l = *++b & ((1 << off) - 1);
out[len_byt] = m ^ (l << rest);
out[max] = (m ^ (l << rest)) & ((1 << len_bit) - 1);
}
}
}
@ -1035,67 +1031,60 @@ ur_bsw64(ur_bsw_t *bsw, uint8_t len, uint64_t val)
}
static inline void
_bsw_bytes_unsafe(ur_bsw_t *bsw, uint64_t len, uint8_t *byt)
_bsw_bytes_unsafe(ur_bsw_t *bsw, const uint64_t len, const uint8_t* src)
{
uint64_t len_byt = len >> 3;
uint8_t len_bit = ur_mask_3(len);
uint64_t fill = bsw->fill;
uint8_t off = bsw->off;
uint8_t *dst = bsw->bytes + fill;
if ( !off ) {
memcpy(bsw->bytes + fill, byt, len_byt);
fill += len_byt;
off = len_bit;
const uint64_t len_byt = len >> 3;
const uint8_t len_bit = ur_mask_3(len);
if ( off ) {
bsw->bytes[fill] = byt[len_byt] & ((1 << off) - 1);
memcpy(dst, src, len_byt);
bsw->fill = fill + len_byt;
if ( len_bit ) {
dst[len_byt] = src[len_byt] & ((1 << len_bit) - 1);
bsw->off = len_bit;
}
}
// the least-significant bits of the input become the
// most-significant bits of a byte in the output stream, and vice-versa
//
else {
uint8_t rest = 8 - off;
uint8_t mask = (1 << rest) - 1;
uint8_t l, m = bsw->bytes[fill];
uint64_t i;
const uint8_t rest = 8 - off;
for ( i = 0; i < len_byt; i++ ) {
l = byt[i] & mask;
bsw->bytes[fill++] = m ^ (l << off);
m = byt[i] >> rest;
if ( rest >= len ) {
uint16_t ful = off + len;
*dst ^= (*src & ((1 << len) - 1)) << off;
if ( ful >> 3 ) {
bsw->fill = fill + 1;
}
// no trailing bits; we need only write the rest of the last byte.
//
// NB: while semantically equivalent to the subsequent block,
// this case must be separate to avoid reading off the end of [byt]
//
if ( !len_bit ) {
bsw->bytes[fill] = m;
bsw->off = ur_mask_3(ful);
}
// trailing bits fit into the current output byte.
//
else if ( len_bit < rest ) {
l = byt[len_byt] & ((1 << len_bit) - 1);
bsw->bytes[fill] = m ^ (l << off);
off += len_bit;
}
// trailing bits extend into the next output byte.
//
else {
l = byt[len_byt] & mask;
bsw->bytes[fill++] = m ^ (l << off);
const uint64_t nel = len - rest;
const uint64_t len_byt = nel >> 3;
const uint8_t len_bit = ur_mask_3(nel);
m = byt[len_byt] >> rest;
*dst++ ^= *src << off;
off = len_bit - rest;
bsw->bytes[fill] = m & ((1 << off) - 1);
for ( uint64_t i = 0; i < len_byt; i++ ) {
dst[i] = (src[i] >> rest) ^ (src[i + 1] << off);
}
{
uint8_t tal = (src[len_byt] >> rest)
^ (( off > len_bit ) ? 0 : (src[len_byt + 1] << off));
dst[len_byt] = tal & ((1 << len_bit) - 1);
}
bsw->fill = fill + len_byt + 1;
bsw->off = len_bit;
}
}
bsw->off = off;
bsw->fill = fill;
bsw->bits += len;
}

2
pkg/urbit/version
View File

@ -1 +1 @@
1.12
1.13