This allows us to check code for syntax errors without relying on
Function(), which can lead to false negatives as certain things are
valid in a function context, but not outside one.
With this, `ntpquery` can adjust the system time without
making it jump.
A fun activity with this in:
0. Boot
1. Run `su`
2. Run `ntpquery -a` to adjust the time offset after boot
(usually around a second)
3. Keep running `ntpquery ; adjtime` to see how the offset
behind NTP and the remaining adjtime both shrink.
adjtime adjustment is large enough to make the time offset
go down by a bit, but we currently lose time quickly enough
that by the time adjtime is done, we've only corrected the
clock about halfway, and not all the way to zero. Goto 2.
So this isn't all that great yet, but I think it's good enough
to think about turning this into a permanently running service next.
It's a thin userland wrapper around adjtime(2). It can be used
to view current pending time adjustments, and root can use it to
smoothly adjust the system time.
As far as I can tell, other systems don't have a userland utility
for this, but it seems useful. Useful enough that I'm adding it to
the lagom build so I can use it on my linux box too :)
> function f(){f()}f()
Uncaught exception: [RuntimeError]: Call stack size limit exceeded
-> f
1234 more calls
-> (global execution context)
> function a(x){if(x>0){a(x-1)}else{throw Error()}}function b(x){if(x>0){b(x-1)}else{a(5)}}function c(){b(2)}c()
Uncaught exception: [Error]
-> a
5 more calls
-> b
-> b
-> b
-> c
-> (global execution context)
You can now do things like "kill -STOP pid" :^)
The getsignalbyname() helper function should probably move to LibC
or somewhere where it can be used by other signal related programs.
`snprintf` returns the number of characters that would have been written
had the buffer been large enough.
It's a common trick to call `snprintf(nullptr, 0, ...)` to measure how
large a buffer has to be.
Thus the return value is not zero but fourteen.
Add an implementation for uniq. While it will be nice in the future to
make more hardened versions of sort and uniq in the future, this
implementation of uniq is compatible with the current version of sort
and its buffer sizes.
This version supports optional input and output files along with stdin
and stdout.
From https://youtu.be/YNSAZIW3EM0?t=1474:
"Hmm... I don't think that name is right! From the perspective of
userspace, this is a file descriptor. File description is what the
kernel internally keeps track of, but as far as userspace is concerned,
he just has a file descriptor. [...] Maybe that name should be changed."
Core::File even has a member of this enum type... called
m_should_close_file_descriptor - so let's just rename it :^)
Ref-counted objects must not be stack allocated. Make DOM::Document's
constructor private to avoid this issue. (I wish we could mark classes
as heap-only..)
Problem:
- `constexpr` functions are decorated with the `inline` specifier
keyword. This is redundant because `constexpr` functions are
implicitly `inline`.
- [dcl.constexpr], §7.1.5/2 in the C++11 standard): "constexpr
functions and constexpr constructors are implicitly inline (7.1.2)".
Solution:
- Remove the redundant `inline` keyword.
test-js now has a --test262-parser-tests option. Modules are skipped for
now, current results:
Test Suites: 1309 failed, 4314 passed, 5623 total
Tests: 1309 failed, 262 skipped, 4052 passed, 5623 total
Files: 5361 total
Time: ~100ms (Lagom) / 600-800ms (Serenity)
For more info, see: https://github.com/tc39/test262-parser-tests
Right now test-js has a hardcoded test root directory when running on
Serenity or will get it based on SERENITY_ROOT otherwise. Now it is
also possible to pass a path to the command which will take precedence
over these mechanisms.
This will also be useful for adding test262 support as those files will
not have a default location.
This implements the transmit time suggestion in (abandoned?)
draft-ietf-ntp-data-minimization. (The other suggestions were already
implemented as far as I can tell.)
Roughly 7% of test-js runtime was spent creating FlyStrings from string
literals. This patch frontloads that work and caches all the commonly
used names in LibJS on a CommonPropertyNames struct that hangs off VM.
This patch causes cp to investigate whether a directory is being copied
into a subdirectory of itself. It uses realpath(3) to ensure that links
do not confound detection.
So far we have three different syntax highlighters for LibJS:
- js's Line::Editor stylization
- JS::MarkupGenerator
- GUI::JSSyntaxHighlighter
This not only caused repetition of most token types in each highlighter
but also a lot of inconsistency regarding the styling of certain tokens:
- JSSyntaxHighlighter was considering TokenType::Period to be an
operator whereas MarkupGenerator categorized it as punctuation.
- MarkupGenerator was considering TokenType::{Break,Case,Continue,
Default,Switch,With} control keywords whereas JSSyntaxHighlighter just
disregarded them
- MarkupGenerator considered some future reserved keywords invalid and
others not. JSSyntaxHighlighter and js disregarded most
Adding a new token type meant adding it to ENUMERATE_JS_TOKENS as well
as each individual highlighter's switch/case construct.
I added a TokenCategory enum, and each TokenType is now associated to a
certain category, which the syntax highlighters then can use for styling
rather than operating on the token type directly. This also makes
changing a token's category everywhere easier, should we need to do that
(e.g. I decided to make TokenType::{Period,QuestionMarkPeriod}
TokenCategory::Operator for now, but we might want to change them to
Punctuation.
In the future all (normal) output should be written by any of the
following functions:
out (currently called new_out)
outln
dbg (currently called new_dbg)
dbgln
warn (currently called new_warn)
warnln
However, there are still a ton of uses of the old out/warn/dbg in the
code base so the new functions are called new_out/new_warn/new_dbg. I am
going to rename them as soon as all the other usages are gone (this
might take a while.)
I also added raw_out/raw_dbg/raw_warn which don't do any escaping,
this should be useful if no formatting is required and if the input
contains tons of curly braces. (I am not entirely sure if this function
will stay, but I am adding it for now.)
Each JS global object has its own "console", so it makes more sense to
store it in GlobalObject.
We'll need some smartness later to bundle up console messages from all
the different frames that make up a page later, but this works for now.
This finally takes care of the kind-of excessive boilerplate code that were the
ctype adapters. On the other hand, I had to link `LibC/ctype.cpp` to the Kernel
(for `AK/JsonParser.cpp` and `AK/Format.cpp`). The previous commit actually makes
sense now: the `string.h` includes in `ctype.{h,cpp}` would require to link more LibC
stuff to the Kernel when it only needs the `_ctype_` array of `ctype.cpp`, and there
wasn't any string stuff used in ctype.
Instead of all this I could have put static derivatives of `is_any_of()` in the
concerned AK files, however that would have meant more boilerplate and workarounds;
so I went for the Kernel approach.
More work on decoupling the general runtime from Interpreter. The goal
is becoming clearer. Interpreter should be one possible way to execute
code inside a VM. In the future we might have other ways :^)
This patch moves the exception state, call stack and scope stack from
Interpreter to VM. I'm doing this to help myself discover what the
split between Interpreter and VM should be, by shuffling things around
and seeing what falls where.
With these changes, we no longer have a persistent lexical environment
for the current global object on the Interpreter's call stack. Instead,
we push/pop that environment on Interpreter::run() enter/exit.
Since it should only be used to find the global "this", and not for
variable storage (that goes directly into the global object instead!),
I had to insert some short-circuiting when walking the environment
parent chain during variable lookup.
Note that this is a "stepping stone" commit, not a final design.
In /proc/PID/fds we get not only file name, but also additional information
about file type, state etc. This commit makes `lsof' command separate these
components. When you are filtering files by file name, only actual file name
is checked (not additional data).
To make it a little clearer what this is for. (This is an RAII helper
class for adding and removing an Interpreter to a VM's list of the
currently active (executing code) Interpreters.)
Taking a big step towards a world of multiple global object, this patch
adds a new JS::VM object that houses the JS::Heap.
This means that the Heap moves out of Interpreter, and the same Heap
can now be used by multiple Interpreters, and can also outlive them.
The VM keeps a stack of Interpreter pointers. We push/pop on this
stack when entering/exiting execution with a given Interpreter.
This allows us to make this change without disturbing too much of
the existing code.
There is still a 1-to-1 relationship between Interpreter and the
global object. This will change in the future.
Ultimately, the goal here is to make Interpreter a transient object
that only needs to exist while you execute some code. Getting there
will take a lot more work though. :^)
Note that in LibWeb, the global JS::VM is called main_thread_vm(),
to distinguish it from future worker VM's.
We can now see at which time a packet was received by the network
adapter, instead of having to measure user time after receiving
the packet in user space.
This means the destination timestamp is no longer affected by in-kernel
queuing delays, which can be tens of milliseconds when the system
is under load.
It also means that if ntpquery grows a message queue that waits on
replies from several requests, the time used processing one response
won't be incorrectly included in the destination timestamp of the
next response (in case two responses arrive at the network adapter
at roughly the same time).
NTP's calculations work better if send and receive latency are
about equal, and this only removes in-kernel queue delays and
context switch delays for the receiving packet. But the two
latencies aren't very equal anyways because $network. Also, maybe
we can add another API for setting the send time in the outgoing
packet in kernel space right before (or when) hitting the network
adapter and use that here too. So this still seems like progress.
Consider the following snippet:
void foo(InputStream& stream) {
if(!stream.eof()) {
u8 byte;
stream >> byte;
}
}
There is a very subtle bug in this snippet, for some input streams eof()
might return false even if no more data can be read. In this case an
error flag would be set on the stream.
Until now I've always ensured that this is not the case, but this made
the implementation of eof() unnecessarily complicated.
InputFileStream::eof had to keep a ByteBuffer around just to make this
possible. That meant a ton of unnecessary copies just to get a reliable
eof().
In most cases it isn't actually necessary to have a reliable eof()
implementation.
In most other cases a reliable eof() is avaliable anyways because in
some cases like InputMemoryStream it is very easy to implement.
It has been possible for a signal to arrive in between us checking
g_interrupted and exiting - sucessfully, even though we were interrupted.
This way, if a signal arrives before we reset the disposition, we
will reliably check for it later; if it arrives afterwards, it'll
kill us automatically.
No difference in practice, but it's tidier and protects us
if we ever use g_interrupted earlier in main -- in that case,
the compiler might chose to keep the variable in a register without
the volatile.
Found by @bugaevc, thanks!
With this, hitting ctrl-c twice in `for i in $(seq 10) { sleep 1 }`
terminates the loop as expected (...well, I'd expect it to quit after
just one ctrl-c, but serenity's shell makes a single ctrl-c only
quit the current loop iteration).
Part of #3419.
Using a pool.ntp.org server seems nicer and more open-source-y,
but until our pool use is approved, let's put in a default value
that works.
(time.google.com serves smeared time instead of doing leap seconds.
pool.ntp.org doesn't serve smeared time. I intend to implement
client-side leap second smearing since nobody likes jumpy timestamps.
For now, we get this for free.)
This fetches information from /var/run/utmp and shows the interactive
sessions in a little list. More things can be added here to make it
more interesting. :^)
To keep track of ongoing terminal sessions, we now have a sort-of
traditional /var/run/utmp file, like other Unix systems.
Unlike other Unix systems however, ours is of course JSON. :^)
The /bin/utmpupdate program is used to update the file, which is
not writable by regular user accounts. This helper program is
set-GID "utmp".
I suspected an error in CircularDuplexStream::read(Bytes, size_t). This
does not appear to be the case, this test case is useful regardless.
The following script was used to generate the test:
import gzip
uncompressed = []
for _ in range(0x100):
uncompressed.append(1)
for _ in range(0x7e00):
uncompressed.append(0)
for _ in range(0x100):
uncompressed.append(1)
compressed = gzip.compress(bytes(uncompressed))
compressed = ", ".join(f"0x{byte:02x}" for byte in compressed)
print(f"""\
TEST_CASE(gzip_decompress_repeat_around_buffer)
{{
const u8 compressed[] = {{
{compressed}
}};
u8 uncompressed[0x8011];
Bytes{{ uncompressed, sizeof(uncompressed) }}.fill(0);
uncompressed[0x8000] = 1;
const auto decompressed = Compress::GzipDecompressor::decompress_all({{ compressed, sizeof(compressed) }});
EXPECT(compare({{ uncompressed, sizeof(uncompressed) }}, decompressed.bytes()));
}}
""", end="")
Addresses #3394 which was caused by dereferencing null `test_name`
when no arguments were given to /bin/tt. Additionally, arguments
other than the defined tests now print usage and exit to avoid
running the default test.
This only queries a single NTP server, only does a point-to-point
request, doens't do any filtering, doesn't display the response
in any useful format, and is generally very bare-bones.
But maybe, over time it can learn to query more servers, do
filtering, run as a service that keeps state over time to
improve filtering, adjust system time, and maybe learn to
run as an NTP server then.
And also mark strlcpy() and strlcat() with __attribute__((warn_unused_result)).
Since our code is warning-free, this ensures we never misuse those functions.
(Or are very sure about doing it when turning off the warning for a particular
piece of code.)
There was a logic mistake in the code that computes the offset between
columns, forgetting to account for the extra characters after a name.
The comment was accurate, but the code didn't match the comment.
Now we have an actual stream implementation that can read arbitrary
(dynamic codes aren't supported yet) deflate encoded data. Even if
the blocks are really large.
And all of that happens with a single buffer of 32KiB. DEFLATE is
amazing!
The fact that a `MarkedValueList` had to be created was just annoying,
so here's an alternative.
This patchset also removes some (now) unneeded MarkedValueList.h includes.
While this _does_ add a point of failure, it'll be a pretty bad day when
google goes down.
And this is unlikely to put a (positive) dent in their incoming
requests, so let's just roll with it until we have our own TLS server.
A malicious caller of ifconfig could have caused the ifr_name field to
lack NUL-termination. I don't think this was an actual problem, though, as
the Kernel always forces NUL-termination by overwriting ifr_name's last byte
with NUL.
However, it feels better to do it properly.
LibJS doesn't store stacks for exception objects, so this
only amends test-common.js's __expect() with an optional
`details` function that can produce a more detailed error
message, and it lets test-js.cpp read and print that
error message. I added the optional details parameter to
a few matchers, most notably toBe() where it now prints
expected and actual value.
It'd be nice to have line numbers of failures, but that
seems hard to do with the current design, and this is already
much better than the current state.
Previously, the implementation would produce one Vector<u8> which
would contain the whole decompressed data. That can be a lot and
even exhaust memory.
With these changes it is still necessary to store the whole input data
in one piece (I am working on this next,) but the output can be read
block by block. (That's not optimal either because blocks can be
arbitrarily large, but it's good for now.)
This lets disasm output contain the symbol names of call and jump
destinations:
8048111: e8 88 38 01 00 call 805b99e <__cxa_atexit>
...
8048150: 74 15 je 8048167 <_start+0x4c>
The latter (the symbol of the current function with an offset) is
arguably more distracting than useful because you usually want to look
at the instruction at the absolute offset in this case, but the former
is very nice to have.
For reasons I do not understand, this cuts the time to run
`disasm /bin/id` in half, from ~1s to ~0.5s.
Let's use the one in AK/NumberFormat.h everywhere.
It has slightly different behavior than some of the copies this
removes, but it's probably nice to have uniform human readable
size outputs across the system.
The SI prefixes "k", "M", "G" mean "10^3", "10^6", "10^9".
The IEC prefixes "Ki", "Mi", "Gi" mean "2^10", "2^20", "2^30".
Let's use the correct name, at least in code.
Only changes the name of the constants, no other behavior change.
I originally defined the bytes() method for the String class, because it
made it obvious that it's a span of bytes instead of span of characters.
This commit makes this more consistent by defining a bytes() method when
the type of the span is known to be u8.
Additionaly, the cast operator to Bytes is overloaded for ByteBuffer and
such.
This is a follow up to #2936 / d3e3b4ae56aa79d9bde12ca1f143dcf116f89a4c.
Affected programs:
- Applications: Browser (Download, View source, Inspect DOM tree, JS
console), Terminal (Settings)
- Demos: Cube, Eyes, Fire, HelloWorld, LibGfxDemo, WebView,
WidgetGallery
- DevTools: HackStudio, Inspector, Profiler
- Games: 2048, Minesweeper, Snake, Solitaire
- Userland: test-web
A few have been left out where manual positioning is done on purpose,
e.g. ClipboardManager (to be close to the menu bar) or VisualBuilder (to
preserve alignment of the multiple application windows).
Decorated Interpreter::call() with [[nodiscard]] to provoke thinking
about the returned value at each call site. This is definitely not
perfect and we should really start thinking about slimming down the
public-facing LibJS interpreter API.
Fixes#3136.
This enables a nice warning in case a function becomes dead code. Also,
in the case of test-crypto.cpp, I took the liberty to add the prefix 'g_'
to the global event loop.
Technically, this can be 'exploited' to set the pgid of an exploiting process
to a near-arbitrary new pgid. This can cause conflicts when assigning future pgids,
destroys the session-boundary, and might confuse future pgid-to-session lookups.
In practice, I can't come up with a way that this causes actual harm.
The symbol name insertion scheme is different from objdump -d's.
Compare the output on Build/Userland/id:
* disasm:
...
_start (08048305-0804836b):
08048305 push ebp
...
08048366 call 0x0000df56
0804836b o16 nop
0804836d o16 nop
0804836f nop
(deregister_tm_clones (08048370-08048370))
08048370 mov eax, 0x080643e0
...
_ZN2AK8Utf8ViewC1ERKNS_6StringE (0805d9b2-0805d9b7):
_ZN2AK8Utf8ViewC2ERKNS_6StringE (0805d9b2-0805d9b7):
0805d9b2 jmp 0x00014ff2
0805d9b7 nop
* objdump -d:
08048305 <_start>:
8048305: 55 push %ebp
...
8048366: e8 9b dc 00 00 call 8056006 <exit>
804836b: 66 90 xchg %ax,%ax
804836d: 66 90 xchg %ax,%ax
804836f: 90 nop
08048370 <deregister_tm_clones>:
8048370: b8 e0 43 06 08 mov $0x80643e0,%eax
...
0805d9b2 <_ZN2AK8Utf8ViewC1ERKNS_6StringE>:
805d9b2: e9 eb f6 ff ff jmp 805d0a2 <_ZN2AK10StringViewC1ERKNS_6StringE>
805d9b7: 90 nop
Differences:
1. disasm can show multiple symbols that cover the same instructions.
I've only seen this happen for C1/C2 (and D1/D2) ctor/dtor pairs,
but it could conceivably happen with ICF as well.
2. disasm separates instructions that do not belong to a symbol with
a newline, so that nop padding isn't shown as part of a function
when it technically isn't.
3. disasm shows symbols that are skipped (due to having size 0)
in parenthesis, separated from preceding and following instructions.
This function did a const_cast internally which made the call side look
"safe". This method is removed completely and call sites are replaced
with ByteBuffer::wrap(const_cast<void*>(data), size) which makes the
behaviour obvious.
This fixes the issue with the exported data having a leading zero,
causing RSA::encrypt to trim the block down, and ruining the encryption.
Fixes#2691 :^)
We were accidentally calling memset() on "addr" (the input char*), not
"dst_addr" (the target struct sockaddr_in), which was causing a simple
"nc localhost 8000" to crash.
Fixes#2908.
