Otherwise, we end up propagating those dependencies into targets that
link against that library, which creates unnecessary link-time
dependencies.
Also included are changes to readd now missing dependencies to tools
that actually need them.
Also do this for Shell.
This greatly simplifies the CMakeLists in Lagom, replacing many glob
patterns with a big list of libraries. There are still a few special
libraries that need some help to conform to the pattern, like LibELF and
LibWebView.
It also lets us remove essentially all of the Serenity or Lagom binary
directory detection logic from code generators, as now both projects
directories enter the generator logic from the same place.
URL had properly named replacements for protocol(), set_protocol() and
create_with_file_protocol() already. This patch removes these function
and updates all call sites to use the functions named according to the
specification.
See https://url.spec.whatwg.org/#concept-url-scheme
This was present in Vector already. Clang-format fixed some const
positions automatically too.
Also removed a now-ambiguous and unnecessary constructor from Shell.
We previously had at least three different implementations for resolving
executables in the PATH, all of which had slightly different
characteristics.
Merge those into a single implementation to keep the behaviour
consistent, and maybe to make that implementation more configurable in
the future.
When |& is typed, stderr will be piped to stdout before the actual
piping happens. This behaves basically like a 2>&1 | (and the
underlying implementation transforms it to that anyway).
Replacement conditions for `requires_argument` have been chosen based
on what would be most convenient for implementing an eventual optional
argument mode.
This prevents us from needing a sv suffix, and potentially reduces the
need to run generic code for a single character (as contains,
starts_with, ends_with etc. for a char will be just a length and
equality check).
No functional changes.
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).
No functional changes.
This commit moves the length calculations out to be directly on the
StringView users. This is an important step towards the goal of removing
StringView(char const*), as it moves the responsibility of calculating
the size of the string to the user of the StringView (which will prevent
naive uses causing OOB access).
StringView was used where possible. Some utilities still use libc
functions which expect null-terminated strings, so String objects were
used there instead.
The lazy resolution mechanism made it so that the variables were linked
together, causing unexpected behaviour:
true
x=$? # expected: x=0
false
echo $x # expected: 0, actual: 1
This moves all code comprehension-related code to a new library,
LibCodeComprehension.
This also moves some types related to code comprehension tasks (such as
autocomplete, find declaration) out of LibGUI and into
LibCodeComprehension.
31ca48e made this default to paths, but now that we have a few sensible
ways to complete things, let's make those work too.
For instance, prior to this `kill <tab>` would've suggested paths, but
now it will suggest processes.
Setting 'allow_commit_without_listing' to false will now make LibLine
show the suggestion before actually committing to it; this is useful for
completions that will replace all the user input, where mistakes can go
unnoticed without some visual cue.
Now that we can resolve these correctly and they're per-suggestion, we
can finally use them for their intended purpose of letting suggestions
overwrite stuff in the buffer.
The shell now expects a JSON object of the form {"kind":<kind>,...} per
line in the standard output of the completion process, where 'kind' is
one of:
- "plain": Just a plain suggestion.
- "program": Prompts the shell to complete a program name starting with
the given "name".
- "proxy": Prompts the shell to act as if a completion for "argv" was
requested.
- "path": Prompts the shell to complete a path given the "base" and
"part" (same as completing part in cwd=base).
We previously allowed globs as match pattern, but for more complex
matching needs, it's nice to have regular expressions.
And as the existing "name a part of the match" concept maps nicely to
named capture groups, we can simply reuse the same code and make groups
with names available in the match body.
For example, when typing `cd <tab>`, the shell will show a list of
files in the current directory. This behavior is similar to typing `cd
./<tab>`.
It makes it easier to `cd` into directories without having to list them
first.
This saves work in places that previously had to create a
`Vector<String>` anyway, or repeatedly cast the char* to a String.
Plus, Strings are nicer than char*. :^)
This feature needs a bit more work, so let's disable it by default.
Note that the shell will still use _complete_foo if it is defined
regardless of this setting.
This commit limits the autocomplete processes to effectively have
readonly access to the fs, and only enough pledges to get the dynamic
loader working.
A program can either respond to `--complete -- some args to complete`
directly, or add a `_complete_<program name>` invokable (i.e. shell
function, or just a plain binary in PATH) that completes the given
command and lists the completions on stdout.
Should such a completion fail or yield no results, we'll fall back to
the previous completion algorithm.
Shell can now use LibLine's `on_paste` hook to more intelligently escape
pasted data, with the following heuristics:
- If the current command is invalid, just pile the pasted string on top
- If the cursor is *after* a command node, escape the pasted data,
whichever way yields a smaller encoding
- If the cursor is at the start of or in the middle of a command name,
paste the data as-is, assuming that the user wants to paste code
- If the cursor is otherwise in some argument, escape the pasted data
according to which kind of string the cursor is in the middle of
(double-quoted, single-quoted or a simple bareword)
Previously would show the list of history items starting from
an index of 0.
This is a bit misleading though. Running `!0` would actually cause
the parser to error out and prevent you from running the command.
The "at most n bytes" behaviour of strncmp is required for this logic to
work, this was overlooked in 5b64abe when converting Strings to
StringViews, which lead to broken autocomplete.
Apologies for the enormous commit, but I don't see a way to split this
up nicely. In the vast majority of cases it's a simple change. A few
extra places can use TRY instead of manual error checking though. :^)
This makes interrupting `sleep 10; echo hi` not print `hi` anymore,
which is the expected behaviour anyway.
Also fixes the problem with fast-running loops "eating" interrupts and
not quitting.
Before this patch, `which ""` or `type ""` would say that the empty
string is `/usr/local/bin/`.
Convert callers to consistently call is_empty() on the returned string
while we're at it, to support eventually removing the is_null() String
state in the future.
Naturally, this means that a command with a failing redirection will
not start, and so will terminate the pipeline (if any).
This also applies to the `exit` run when the shell is closed, fixing a
fun bug there as well (thanks to Discord user Salanty for pointing that
out) where closing the terminal (i.e. I/O error on the tty) with a
failing `exit` command would make the shell retry executing `exit` every
time, leading to an eventual stack overflow.
Whenever the prompt is printed, we write a line's worth of space
characters to the terminal to ensure that the prompt ends up on a new
line even if there is dangling output on the current line.
We write these to the stderr, which is unbuffered, so each putc() call
would come with the overhead of a system call. Let's use a buffer
+ fwrite() instead, since heap allocation is much faster.
Previously we were simply ignoring the empty entry in '{,x}', making it
resolve to a list with a single element '(x)', this commit makes that
work as expected and resolve to '("" x)'.
This option is already enabled when building Lagom, so let's enable it
for the main build too. We will no longer be surprised by Lagom Clang
CI builds failing while everything compiles locally.
Furthermore, the stronger `-Wsuggest-override` warning is enabled in
this commit, which enforces the use of the `override` keyword in all
classes, not just those which already have some methods marked as
`override`. This works with both GCC and Clang.
This isn't a complete conversion to ErrorOr<void>, but a good chunk.
The end goal here is to propagate buffer allocation failures to the
caller, and allow the use of TRY() with formatting functions.
This commit makes the Shell check for errors after a node is run(), and
prevents further execution by unwinding until the error is cleared.
Fixes#10649.
When parse_expression looks at '$((', there are two ways it can end up
in parse_expression again, three consumed characters later. All these
ways fail, so what happened was that the parser tried all possible
combinations, hence taking potentially an exponential amount of time.
1. parse_evaluate swallows the '$(', a new invocation of
parse_expression swallows the other '(', and through
parse_list_expression we're at another parse_expression.
2. parse_evaluate swallows the '$(', but returns a SyntaxError.
parse_expression used to not recognize the error, and treated it as a
regular AST node, calling into read_concat, then a new invocation of
parse_expression swallows the other '(', and through
parse_list_expression we're at another parse_expression.
Fixes#10561.
Found by OSS Fuzz, long-standing issue
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28113
...while capturing its standard output.
As `$()` is an invalid construct, execute nodes are not supposed to
capture the output of no command being run; but it is possible to create
empty commands such as CastToCommand(Redirection(...)) or similar.
Make this a hard error instead of an unescapable select().
This was noticed in #10432, which should now error out like so:
```
Error: Cannot capture standard output when no command is being executed
0| $(<$file)
~~~~~^^^^^^^^^
1|
```
Our existing implementation did not check the element type of the other
pointer in the constructors and move assignment operators. This meant
that some operations that would require explicit casting on raw pointers
were done implicitly, such as:
- downcasting a base class to a derived class (e.g. `Kernel::Inode` =>
`Kernel::ProcFSDirectoryInode` in Kernel/ProcFS.cpp),
- casting to an unrelated type (e.g. `Promise<bool>` => `Promise<Empty>`
in LibIMAP/Client.cpp)
This, of course, allows gross violations of the type system, and makes
the need to type-check less obvious before downcasting. Luckily, while
adding the `static_ptr_cast`s, only two truly incorrect usages were
found; in the other instances, our casts just needed to be made
explicit.
And also try_create<T> => try_make_ref_counted<T>.
A global "create" was a bit much. The new name matches make<T> better,
which we've used for making single-owner objects since forever.
Only one place used this argument and it was to hold on to a strong ref
for the object. Since we already do that now, there's no need to keep
this argument around since this can be easily captured.
This commit contains no changes.
The new Statistics utility is now used when calling 'time -n' to get
some more information of the timings. For now only the standard
deviation is given in addition to the average.
This commit completely undos #9645 because everything that touched moved
into AK::Statistics.
This kinda sorta addresses the Shell side of #9655, however the fact
that `chdir` (and most other syscalls that take paths) are artifically
limited to a length of PATH_MAX remains.
You can now specify a number of iterations when timing a command.
The default value is 1 and behaves exactly as before.
If the iteration count is greater than 1, the command will be executed
that many times, and then you get a little timing report afterwards with
the average runtime per iteration, and also the average runtime
excluding the very first iteration. (Excluding the first iteration is
useful when it's slowed down by cold caches, etc.)
This is something I've been doing manually forever (running `time foo`
and then eyeballing the results to headmath an average) and this makes
that whole process so much nicer. :^)
This is primarily to be able to remove the GenericLexer include out of
Format.h as well. A subsequent commit will add AK::Result to
GenericLexer, which will cause naming conflicts with other structures
named Result. This can be avoided (for now) by preventing nearly every
file in the system from implicitly including GenericLexer.
Other changes in this commit are to add the GenericLexer include to
files where it is missing.
Heredocs have a different parse end condition than double-quoted
strings. parse_doublequoted_string_inner would assume that a string
would always end in a double quote, so let's generalize it to
parse_string_inner and have it take a StringEndCondition enum which
specifies how the string terminates.
That can happen because of anyone sending the process a SIGCONT.
Fixes an issue where continuing a process launched by the shell from
the System Monitor would cause the shell to spin on waitpid().
Since this is always set to true on the non-default constructor and
subsequently never modified, it is somewhat pointless. Furthermore,
there are arguably no invalid relative paths.
This will cause a problem when `NonnullRefPtr<T>::operator T*` will be
declared as RETURNS_NONNULL. Clang emits a warning for this pointless
null check, which breaks CI.
This commit converts naked `new`s to `AK::try_make` and `AK::try_create`
wherever possible. If the called constructor is private, this can not be
done, so we instead now use the standard-defined and compiler-agnostic
`new (nothrow)`.
This is so they can find their associated resources and it's
the same behavior as in Lagom.
This also required changing some tests so that they could
write their resources in a writable location.
Some of the code assumed that chars were always signed while that is
not the case on ARM hosts.
Also, some of the code tried to use EOF (-1) in a way similar to what
fgetc() does, however instead of storing the characters in an int
variable a char was used.
While this seemed to work it also meant that character 0xFF would be
incorrectly seen as an end-of-file.
Careful reading of fgetc() reveals that fgetc() stores character
data in an int where valid characters are in the range of 0-255 and
the EOF value is explicitly outside of that range (usually -1).
And use them to highlight javascript in HTML source.
This commit also changes how TextDocumentSpan::data is interpreted,
as it used to be an opaque pointer, but everyone stuffed an enum value
inside it, which made the values not unique to each highlighter;
that field is now a u64 serial id.
The syntax highlighters don't need to change their ways of stuffing
token types into that field, but a highlighter that calls another
nested highlighter needs to register the nested types for use with
token pairs.
This changes the Shell syntax highlighter to conform to the now-fixed
rendering of syntax highlighting spans in GUI::TextEditor.
This also adds some debug output to make debugging easier.
This replaces ctype.h with CharacterType.h everywhere I could find
issues with narrowing conversions. While using it will probably make
sense almost everywhere in the future, the most critical places should
have been addressed.
We had two functions for doing mostly the same thing. Combine both
of them into String::find() and use that everywhere.
Also add some tests to cover basic behavior.
Follow-on to #7337. Been seeing other CI test failures that point to
these temp directories, so let's just move all of them to /tmp. I'm sure
someone will write ext2fs stress tests later :^)
Example:
/usr/Tests/Shell/control-structure-as-command.sh
Core::Socket: Failed to connect() to /tmp/portal/inspectables: ...
+ rm -rf shell-test 2>/dev/null
+ mkdir shell-test
Error: The action has timed out.
Since applications using Core::EventLoop no longer need to create a
socket in /tmp/rpc/, and also don't need to listen for incoming
connections on this socket, we can remove a whole bunch of pledges!
This patch adds a new flag called history_dirty to Line::Editor that is
set when history is added to but written. Applications can leverage
this flag to write history only when it changes. This patch adds an
example usage of this functionality to Shell, which will now only save
the history when it is dirty.
Instead of the previous only-escape-with-backslashes, extend the
escaping to one of:
- No escape
- Escape with backslash
- Escape with "\xhh" if control character that isn't easily represented
as \X
- Escape with "\uhhhhhhhh" if unicode character that is too big to
represent as "\xhh".
Fixes#6986.
This commit makes the shell:
- highlight executables in the current directory as invalid, unless an
explicit `./' is given (so, `./foo` isn't red, but `foo` is)
- not suggest executables in the current directory unless explicitly
requested (by prepending `./`)
- not attempt to run an executable in the current directory that has
been invoked as a program name and failed execvp().
Note that `./foo` is still executed because it's not invoked as
a name, but rather as a path.
Fixes the other half of #6774.
This bit of code was kept unmodified since it was first implemented,
and I'm not entirely convinced that it ever actually worked :P
This commit updates the code to use "modern" classes and constructs,
and fixes an issue where the shebang would still contain the '#!'
when it was passed to execvp().
Fixes#6774.
Otherwise we would end up trying to parse the same heredoc entry, if it
contained a sequence terminated by a newline.
e.g. `<<-x\n$({` would attempt to read a heredoc entry after `x`, and
then after `{` while inside the first heredoc entry.
To make this work, we can simply empty the instance vector and keep the
state on the stack.
Issue found through oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=33852
Closes#4283.
Heredocs are implemented in a way that makes them feel more like a
string (and not a weird redirection, a la bash).
There are two tunables, whether the string is dedented (`<<-` vs `<<~`)
and whether it allows interpolation (quoted key vs not).
To the familiar people, this is how Ruby handles them, and I feel is the
most elegant heredoc syntax.
Unlike the oddjob that is bash, heredocs are treated exactly as normal
strings, and can be used _anywhere_ where a string can be used.
They are *required* to appear in the same order as used after a newline
is seen when parsing the sequence that the heredoc is used in.
For instance:
```sh
echo <<-doc1 <<-doc2 | blah blah
contents for doc1
doc1
contents for doc2
doc2
```
The typical nice errors are also implemented :^)
Some nodes (such as heredocs) cannot be validated immediately, so the
entire tree will need to be revalidated if we don't allow mutating
syntax errors.
We had some inconsistencies before:
- Sometimes "The", sometimes "the"
- Sometimes trailing ".", sometimes no trailing "."
I picked the most common one (lowecase "the", trailing ".") and applied
it to all copyright headers.
By using the exact same string everywhere we can ensure nothing gets
missed during a global search (and replace), and that these
inconsistencies are not spread any further (as copyright headers are
commonly copied to new files).
Previously this didn't work:
$ cd -- /usr
Invalid path '--'
This path fixes this issue and removes the unnecessary else
branch because we're already using realpath() later on to resolve
relative paths.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.
See: https://spdx.dev/resources/use/#identifiers
This was done with the `ambr` search and replace tool.
ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
We intentionally skimp out on checking isatty() before them to cut down
on syscalls, so we should also accept the errors and just let them be.
Closes#6471.
This flag warns on classes which have `virtual` functions but do not
have a `virtual` destructor.
This patch adds both the flag and missing destructors. The access level
of the destructors was determined by a two rules of thumb:
1. A destructor should have a similar or lower access level to that of a
constructor.
2. Having a `private` destructor implicitly deletes the default
constructor, which is probably undesirable for "interface" types
(classes with only virtual functions and no data).
In short, most of the added destructors are `protected`, unless the
compiler complained about access.
This makes commands like `foo 2>&1 | bar` behave as expected (which is
to pipe both stdout and stderr of `foo` to stdin of `bar`).
Previously, this would've piped stderr of `foo` into stdout, and the
stdout of `foo` into the stdin of `bar`.
This fixes `fg` and `bg` causing the shell to go into an infinite loop
of trying to `waitpid` until some current job changes state.
a.k.a. "Fix Shell backgrounding, yet again!" :P
Now a variable may have an optional slice (only _one_ slice), which can
also use negative indices to index from the end.
This works on both lists and strings.
The contents of the slice have the same semantics as brace expansions.
For example:
```sh
$ x=(1 2 3 4 5 6)
$ echo $x[1..3] # select indices 1, 2, 3
2 3 4
$ echo $x[3,4,1,0] # select indices 3, 4, 1, 0 (in that order)
4 5 2 1
$ x="Well Hello Friends!"
$ echo $x[5..9]
Hello
```
This commit adds a few basic variable substitution operations:
- length
Find the length of a string or a list
- length_across
Find the lengths of things inside a list
- remove_{suffix,prefix}
Remove a suffix or a prefix from all the passed values
- regex_replace
Replace all matches of a given regex with a given template
- split
Split the given string with the given delimiter (or to its
code points if the delimiter is empty)
- concat_lists
concatenates any given lists into one
Closes#4316 (the ancient version of this same feature)
Non-interactive shells (i.e. when running scripts) do not need this
functionality, so they are a boatload of wasted time.
This significantly reduces the script startup and shutdown times when
there are lots of executables in PATH or lots of entries in the history.
This is basically just for consistency, it's quite strange to see
multiple AK container types next to each other, some with and some
without the namespace prefix - we're 'using AK::Foo;' a lot and should
leverage that. :^)
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)
Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.
We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
Thanks to @trflynn89 for the neat implicit consteval ctor trick!
This allows us to basically slap `CheckedFormatString` on any
formatting function, and have its format argument checked at compiletime.
Note that there is a validator bug where it doesn't parse inner replaced
fields like `{:~>{}}` correctly (what should be 'left align with next
argument as size' is parsed as `{:~>{` following a literal closing
brace), so the compiletime checks are disabled on these temporarily by
forcing them to be StringViews.
This commit also removes the now unused `AK::StringLiteral` type (which
was introduced for use with NTTP strings).
