liljencrantz/crush
1
1
Fork 0
mirror of https://github.com/liljencrantz/crush.git synced 2024-10-03 22:07:10 +03:00
Code Issues Projects Releases Wiki Activity

Compare commits

base: liljencrantz:21c5395679edea5f0604c14550594d2bb2effaf1
Branches Tags
liljencrantz:master
liljencrantz:wip
liljencrantz:completion
...
compare: liljencrantz:e177a02fd592f3c563e639d5e5afca26bb232879
Branches Tags
liljencrantz:master
liljencrantz:wip
liljencrantz:completion

10 Commits
21c5395679 ... e177a02fd5

Author SHA1 Message Date
Axel Liljencrantz
e177a02fd5 AST/parsing cleanup 2022-09-06 01:18:00 +02:00
Axel Liljencrantz
8a94efe6b6 Naming 2022-09-05 15:59:20 +02:00
Axel Liljencrantz
60fd099edc Minor refactoring 2022-09-05 15:32:31 +02:00
Axel Liljencrantz
cd2710c8da Fix pup invocation bug 2022-09-03 11:23:06 +02:00
Axel Liljencrantz
14383e2d5d Support streaming globs 2022-09-02 22:10:27 +02:00
Axel Liljencrantz
8830644a66 Drop weird file fallback feature 2022-09-02 21:20:35 +02:00
Axel Liljencrantz
8d96142ac9 Change syntax for the whole language... :-/ 2022-09-02 21:03:01 +02:00
Axel Liljencrantz
ecba86aaac Simplify command invocation code 2022-09-01 19:11:56 +02:00
Axel Liljencrantz
2997cfb062 Simplify command invocation code 2022-08-31 18:36:54 +02:00
Axel Liljencrantz
11e09d09c5 Fix various context bugs 2022-08-31 18:06:19 +02:00
105 changed files with 2221 additions and 1734 deletions
Show Stats Expand all files Collapse all files

1210
Cargo.lock generated
View File

File diff suppressed because it is too large Load Diff

1
Cargo.toml
View File

@ -56,6 +56,7 @@ psutil = "3.2.2"
num-format = { version = "0.4", features = ["with-system-locale"] }
unicode-width = "0.1.5"
os_pipe = "0.9.2"
uptime_lib = "0.2.2"
[target.'cfg(target_os = "linux")'.dependencies]
dbus = "0.8.4"

14
docs/config.md
View File

@ -39,14 +39,14 @@ Crush code in the interactive Crush prompt. Assign ANSI color codes
to the various token types of Crush to make your terminal more closely
resemble a Christmas tree:
| Name | Description |
| --- | --- |
| Name | Description |
| --- |----------------------------------------------------------|
| `operator` | All the different Crush operators, such as `neg` and `+` |
| `numeric_literal` | Integer and floating point literals, such as `6` |
| `string_literal` | String literals, like `"Burrow"` |
| `file_literal` | File literals, like `'Cargo.toml'` |
| `label` | Variables and members, like `global` |
| `field` | Field definitions, such as `^name` |
| `numeric_literal` | Integer and floating point literals, such as `6` |
| `string_literal` | String literals, like `"Burrow"` |
| `file_literal` | File literals, like `'Cargo.toml'` |
| `label` | Variables and members, like `$global` |
| `field` | Field definitions, such as `name` |
The `term` namespace contains useful constants containing ANSI color codes.
A configuration example:

4
docs/misc.md
View File

@ -13,8 +13,8 @@ what I felt was a more suitable type system.
## Similarity to Nushell
On the surface, Crush looks identical to nushell, but less polished. Crush lacks
syntax highlighting, tab completion and has a worse screen rendering. But that
On the surface, Crush looks identical to nushell, but less polished. Crush has
less polished syntax highlighting, tab completion and screen rendering. But that
is because the focus of Crush right now is to create a well defined, powerful
and convenient language that supports things like arithmetic operations,
closures, loops and flow control while remaining useful for interactive use.

105
docs/overview.md
View File

@ -36,7 +36,7 @@ a Rush channel. It is not understood by the command as a series of bytes, but as
a table of rows, and Crush provides you with SQL-like commands to sort, filter,
aggregate and group rows of data.
crush# ll | sort ^size
crush# ll | sort size
user size modified type file
fox 31 2019-10-03 13:43:12 +0200 file .gitignore
fox 75 2020-03-07 17:09:15 +0100 file build.rs
@ -44,7 +44,7 @@ aggregate and group rows of data.
fox 711 2019-10-03 14:19:46 +0200 file crush.iml
...
crush# ll | where {type == "directory"}
crush# ll | where {$type == "directory"}
user size modified type file
fox 4_096 2019-11-22 21:56:30 +0100 directory target
fox 4_096 2020-02-22 11:50:12 +0100 directory tests
@ -84,10 +84,10 @@ commands all work like you'd expect:
crush# ls | json:to ./listing.json
# Read the file Cargo.toml as a toml file, and extract the dependencies-field
crush# toml:from Cargo.toml | member ^dependencies
crush# toml:from Cargo.toml | member dependencies
# Fetch a web page and write it to a file
http "https://isitchristmas.com/" | member ^body | bin:to ./isitchristmas.html
http "https://isitchristmas.com/" | member body | bin:to ./isitchristmas.html
```
If you don't supply an input file to any of the deserializer commands,
@ -109,7 +109,7 @@ crush# list:of "carrot" "carrot" "acorn" | json:to
One of the Crush serializers, Pup, is a native file format for Crush. The
Pup-format is protobuf-based, and its schema is available
[here](src/crush.proto). The advantage of Pup is that all crush types,
[here](../src/crush.proto). The advantage of Pup is that all crush types,
including classes and closures, can be losslessly serialized into this format.
But because Pup is Crush-specific, it's useless for data sharing to
other languages.
@ -142,7 +142,7 @@ are false.
The `and` and `or` operators are used to combine logical expressions:
crush# false or true
crush# $false or $true
true
crush# if (./tree:exists) and ((./tree:stat):is_file) {echo "yay"}
@ -183,16 +183,17 @@ current working directory as a single element of the `file` type.
### Variables of any type
Variables must be declared (using the `:=` operator) before use.
Use the $ sigil to refer to variables. Variables must be declared
(using the `:=` operator) before use.
crush# some_number := 4 # The := operator declares a new variable
crush# some_number * 5
crush# $some_number := 4 # The := operator declares a new variable
crush# $some_number * 5
20
Once declared, a variable can be reassigned to using the `=` operator.
crush# some_number = 6
crush# some_number * 5
crush# $some_number = 6
crush# $some_number * 5
30
Like in any sane programming language, variables can be of any type supported by
@ -218,13 +219,14 @@ the other or a combination of both. The following three invocations are equivale
It is quite common to want to pass boolean arguments to commands, which is why
Crush has a special shorthand syntax for it. Passing in `--foo` is equivalent
to passing in `foo=true`.
to passing in `foo=$true`.
### Subshells
Sometimes you want to use the output of one command as an *argument* to another
command, just like a subshell in e.g. bash. This is different from using the
output as the *input*, and is done by putting the command within parenthesis (`()`), like so:
command, just like a subshell in e.g. bash. This is different from what a pipe does,
which is using the output as the *input*, and is done by putting the command within
parenthesis (`()`), like so:
crush# echo (pwd)
@ -233,14 +235,14 @@ output as the *input*, and is done by putting the command within parenthesis (`(
In Crush, braces (`{}`) are used to create a closure. Assigning a closure to a
variable is how you create a function.
crush# print_greeting := {echo "Hello"}
crush# $print_greeting := {echo "Hello"}
crush# print_greeting
Hello
Any named arguments passed when calling a closure and added to the local scope
of the invocation:
crush# print_a := {echo a}
crush# $print_a := {echo $a}
crush# print_a a="Greetings"
Greetings
@ -251,13 +253,13 @@ The following closure requires the caller to supply the argument `a`, and allows
the caller to specify the argument `b`, which must by of type integer. If the
caller does not specify it, it falls back to a default value of 7.
crush# print_things := {|a b: integer = 7|}
crush# print_things := {|a b: $integer = 7|}
Additionally, the `@` operator can be used to create a list of all unnamed
arguments, and the `@@` operator can be used to create a list of all named
arguments not mentioned elsewhere in the parameter list.
crush# print_everything := {|@unnamed @@named| echo "Named" named "Unnamed" unnamed}
crush# print_everything := {|@unnamed @@named| echo "Named" $named "Unnamed" $unnamed}
The `@` and `@@` operators are also used during command invocation to perform
the mirrored operation. The following code creates an `lss` function that calls
@ -271,7 +273,7 @@ the `select` command to only show one column from the output.
Crush comes with a variety of types:
* lists of any type,
* dicts of any pair of types type, (Some types can not be used as keys!)
* dicts of a pair of types, (Some types can not be used as keys!)
* strings,
* regular expressions,
* globs,
@ -295,15 +297,15 @@ commands.
When playing around with Crush, the `help` and `dir`commands are useful. The
former displays a help messages, the latter lists the content of a value.
crush# help
crush# help $sort
sort column:field
Sort input based on column
Example:
ps | sort ^cpu
crush# dir list
ps | sort cpu
crush# dir $list
[type, truncate, remove, clone, of, __call__, __setitem__, pop, push, empty, len, peek, new, clear]
### The content of your current working directory lives in your namespace
@ -347,21 +349,21 @@ what you'd expect.
If you assign the output of the find command to a variable like so:
crush# all_the_files := (find /)
crush# $all_the_files := (find /)
What will really be stored in the `all_the_files` variable is simply a stream. A
small number of lines of output will be eagerly evaluated, before the thread
executing the find command will start blocking. If the stream is consumed, for
example by writing
crush# all_the_files
crush# $all_the_files
then all hell will break loose on your screen as tens of thousands of lines are
printed to your screen.
Another option would be to pipe the output via the head command
crush# all_the_files | head 1
crush# $all_the_files | head 1
Which will consume one line of output from the stream. This command can be
re-executed until the stream is empty.
@ -369,10 +371,10 @@ re-executed until the stream is empty.
### More SQL-like data stream operations
Crush features many commands to operate om arbitrary streams of data using a
SQL-like syntax. These commands use field-specifiers like `^foo` to specify
SQL-like syntax. These commands use field-specifiers like `foo` to specify
columns in the data stream that they operate on:
ps | where {user == "root"} | group ^status proc_per_status={count} | sort ^proc_per_status
ps | where {$user == "root"} | group status proc_per_status={count} | sort proc_per_status
status proc_per_status
Idle 108
Sleeping 170
@ -395,7 +397,7 @@ operator instead. `?` is still used for single character wildcards.
The operator `%%` is used for performing globbing recursively into subdirectories.
Another way of looking ath the same syntax is to say that `%` and `?` match any
character except `/`, whereas `%%` also matches `/`.
character except `/`, whereas `%%` matches any character including `/`.
# Count the number of lines of rust code in the crush source code
crush# lines src/%%.rs|count
@ -506,8 +508,8 @@ computers memory, and even infinite data sets.
But sometimes, streaming data sets are inconvenient, especially if one wants to
use the same dataset twice.
crush# files := ls
crush# files
crush# $files := ls
crush# $files
user size modified type file
fox 1_307 2020-03-26 01:08:45 +0100 file ideas
fox 4_096 2019-11-22 21:56:30 +0100 directory target
@ -522,17 +524,17 @@ use the same dataset twice.
fox 8_382 2020-03-29 00:54:13 +0100 file todo
fox 75 2020-03-07 17:09:15 +0100 file build.rs
fox 711 2019-10-03 14:19:46 +0200 file crush.iml
crush# files
crush# $files
Notice how there is no output the second time the content of the `files` variable is
displayed, because the table_input_stream has already been consumed.
Enter the materialize command, which takes any value and recursively converts
all transient (table_input_stream and binary_stream) components into an equivalent
in-memory form (stable, and binary, respectively).
in-memory form (table, and binary, respectively).
crush# materialized_files := (ls|materialize)
crush# materialized_files
crush# $materialized_files := (ls|materialize)
crush# $materialized_files
user size modified type file
fox 1307 2020-03-26 01:08:45 +0100 file ideas
fox 4096 2019-11-22 21:56:30 +0100 directory target
@ -547,7 +549,7 @@ in-memory form (stable, and binary, respectively).
fox 8_382 2020-03-29 00:54:13 +0100 file todo
fox 75 2020-03-07 17:09:15 +0100 file build.rs
fox 711 2019-10-03 14:19:46 +0200 file crush.iml
crush# materialized_files
crush# $materialized_files
user size modified type file
fox 1307 2020-03-26 01:08:45 +0100 file ideas
fox 4096 2019-11-22 21:56:30 +0100 directory target
@ -563,7 +565,7 @@ in-memory form (stable, and binary, respectively).
fox 75 2020-03-07 17:09:15 +0100 file build.rs
fox 711 2019-10-03 14:19:46 +0200 file crush.iml
When the `table_input_stream` is materialized into a `table`, it can be displayed
When the `table_input_stream` is materialized into a `table`, it can be accessed
multiple times.
### Flow control
@ -612,13 +614,10 @@ non-interactive commands work as expected:
Crush features several shortcuts to make working with external commands easier.
* Firstly, subcommands like `git status` are mapped into method calls like
`git:status`. That way you do not have to quote the subcommand name, e.g.
`git "status"`.
* Secondly, named arguments are transparently translated into options. Single
* Named arguments are transparently translated into options. Single
character argument names are turned into options with a single hyphen, and
multi-character argument names are turned into GNU style long options with
two hyphens, e.g. `git:commit m="hello"` is converted into
two hyphens, e.g. `git commit m="hello"` is converted into
`git commit -m "hello"` and `git:commit message="hello"` is converted into
`git commit --message "hello"`.
* Thirdly, named arguments with a value of boolean true are simply turned into
@ -661,26 +660,26 @@ To run a closure on multiple remote hosts, use `remote:pexec` instead.
You can create custom types in Crush, by using the class command:
Point := (class)
$Point := (class)
Point:__init__ = {
|x:float y:float|
this:x = x
this:y = y
$Point:__init__ = {
|$x:$float $y:$float|
$this:x = $x
$this:y = $y
}
Point:len = {
$Point:len = {
||
math:sqrt this:x*this:x + this:y*this:y
math:sqrt $this:x*$this:x + $this:y*$this:y
}
Point:__add__ = {
|other|
Point:new x=(this:x + other:x) y=(this:y + other:y)
$Point:__add__ = {
|$other|
Point:new x=($this:x + $other:x) y=($this:y + $other:y)
}
p := (Point:new x=1.0 y=2.0)
p:len
$p := (Point:new x=1.0 y=2.0)
$p:len
Crush supports single inheritance (by passing in the parent to the class
command). The class command will create a new struct, that contains a method

37
docs/syntax.md
View File

@ -37,7 +37,7 @@ input and produce a single value as output. The input and output of a command
is passed down via a so called pipeline:
```shell script
ps | sort ^cpu
ps | sort cpu
```
Many commands consume and produce table streams as input and output. These commands
@ -54,18 +54,25 @@ A character sequence enclosed within double quotes become a string literal value
e.g. `"hello"`.
A character sequence enclosed within single quotes become a file literal, e.g.
`'Cargo.yaml'`, i.e. a value of the type `file`. A character sequence that begins
with a dot (`.`), a tilde (`~`) or slash (`/`) is also interpreted as a file literal, e.g.
`./Cargo.yam` or `/etc`.
`'Cargo.yaml'`, i.e. a value of the type `file`. An unquoted character sequence
that contains a dot (`.`) or a slash ('/'), or begins with a tilde (`~`) is also
interpreted as a file literal, e.g. `Cargo.yaml` or `~/.ssh`.
A character sequence starting with a caret (`^`) becomes a field literal, e.g.
`^user`. Field literals are used by some Crush commands to specify specific columns
in a stream. For example, to tell the sort command to sort the input stream by the
column `user`, one would write `sort ^user`.
Other character sequences are interpreted as a variable lookup. Commands live in
A character sequence starting with a caret (`$`) is intepreted as a variable
lookup. The first sequewnce in a command (i.e. the command name) is interpreted
as a variable lookup even without the leading `$`. Commands live in
the same namespace as all other variables.
All other unquoted character sequences are field literals, e.g.
`user` or `hat`. Field literals are basically short strings. They
are used by some Crush commands to specify specific columns
in a stream. For example, to tell the sort command to sort the
input stream by the column `user`, one would write `sort user`.
Commands that expect files as argument, like `cd` or `find`,
generally accept fields as files. They also accept globs and
regular expression literals (see below), and expand them into
file names.
## Operators
Crush features a number of operators to enable users to write mathematical
@ -95,8 +102,9 @@ command. To do this, simply put the command within parenthesis:
if the command that you run in a substitution returns a stream, the outer command will
be run in parallel with the substitution and may in fact finish running first.
This example will create a command that locates all the files in your computer, and assign
the output stream to a variable.
This example will create a command that locates all the files in your computer, and assigns
the output stream to a variable. The `find` command in this example will block because there
is nothing reading its output.
```shell script
all_the_files := (find /)
@ -113,7 +121,8 @@ A few of the namespaces in Crush are:
* the `crush` namespace, which contains runtime information about the current shell
as well as methods and variables that allow you to reconfigure it,
* the `fd` namespace, which contains information about file descriptors, e.g. all
open network sockets, unix sockets and open files.
* the `user` namespace which contains information about all the users of this system,
* the `fd` namespace, which contains information about open file descriptors, e.g. all
open network sockets, unix sockets and open files, and
* the `host` namespace, which contains information about the current host, including
host name, CPU status, memory usage and operating system.

14
signature/src/lib.rs
View File

@ -79,7 +79,7 @@ fn extract_type(ty: &Type) -> SignatureResult<(&'static str, Vec<&'static str>)>
"Command" => "Command",
"Duration" => "Duration",
"Struct" => "Struct",
"Field" => "Field",
"Symbol" => "Symbol",
"Value" => "Value",
"Stream" => "Stream",
"Number" => "Number",
@ -169,7 +169,7 @@ fn simple_type_to_value(simple_type: &str) -> TokenStream {
"char" => quote! {crate::lang::value::Value::String(_value)},
"Command" => quote! {crate::lang::value::Value::Command(_value)},
"Duration" => quote! {crate::lang::value::Value::Duration(_value)},
"Field" => quote! {crate::lang::value::Value::Field(_value)},
"Symbol" => quote! {crate::lang::value::Value::Symbol(_value)},
"Struct" => quote! {crate::lang::value::Value::Struct(_value)},
"Stream" => quote! {_value},
"Value" => quote! {_value},
@ -192,7 +192,7 @@ fn simple_type_to_value_type(simple_type: &str) -> TokenStream {
"char" => quote! {crate::lang::value::ValueType::String},
"Command" => quote! {crate::lang::value::ValueType::Command},
"Duration" => quote! {crate::lang::value::ValueType::Duration},
"Field" => quote! {crate::lang::value::ValueType::Field},
"Symbol" => quote! {crate::lang::value::ValueType::Symbol},
"Struct" => quote! {crate::lang::value::ValueType::Struct},
_ => quote! {crate::lang::value::ValueType::Any},
}
@ -213,7 +213,7 @@ fn simple_type_to_value_description(simple_type: &str) -> &str {
"char" => "string",
"Command" => "command",
"Duration" => "duration",
"Field" => "field",
"Symbol" => "symbol",
"Value" => "any value",
"Stream" => "stream",
"Struct" => "Struct",
@ -241,7 +241,7 @@ fn simple_type_to_mutator(simple_type: &str, allowed_values: &Option<Ident>) ->
"Stream" => {
quote! {
crate::lang::errors::mandate_argument(
_value.stream(),
_value.stream()?,
"Expected a type that can be streamed",
_location)?,
}
@ -299,7 +299,7 @@ fn simple_type_dump_list(simple_type: &str) -> &str {
"ValueType" => "dump_type",
"f64" => "dump_float",
"Value" => "dump_value",
"Field" => "dump_field",
"Symbol" => "dump_symbol",
_ => panic!("Unknown type"),
}
}
@ -320,7 +320,7 @@ fn parse_type_data(
let (type_name, args) = extract_type(ty)?;
match type_name {
"i128" | "bool" | "String" | "char" | "ValueType" | "f64" | "Command" | "Duration"
| "Field" | "Value" | "usize" | "i64" | "u64" | "i32" | "u32" | "Stream" | "Struct" => {
| "Symbol" | "Value" | "usize" | "i64" | "u64" | "i32" | "u32" | "Stream" | "Struct" => {
if !args.is_empty() {
fail!(ty.span(), "This type can't be paramterizised")
} else {

4
src/crush.proto
View File

@ -24,7 +24,7 @@ message Element {
Closure closure = 9; // A Value::Command that is a closure
bytes binary = 10; // A Value::binary
Duration duration = 11; // A Value::Duration
uint64 field = 12; // A Value::Field
string symbol = 12; // A Value::Symbol
string glob = 13; // A Value::Glob
string regex = 14; // A Value::RegEx
Scope user_scope = 15; // A Value::Scope
@ -204,7 +204,7 @@ message Type {
COMMAND = 4;
BINARY = 5;
DURATION = 6;
FIELD = 7;
SYMBOL = 7;
GLOB = 8;
REGEX = 9;
SCOPE = 10;

20
src/crushlib/lls.crush
View File

@ -1,23 +1,21 @@
ls := {
|sort_by:field=^file @args|
$ls := {
|@ $args |
"List names of files non-recursively"
" Unlike find and ll, ls only shows you the names of files.
sort_by can be one of ^user, ^size, ^modified, ^type or ^file.
Example:
ls / sort_by=^size"
find recursive=false @args | sort sort_by| select ^file
ls /"
find recursive=$false @ $args | sort file| select file
}
ll := {
|sort_by:field=^file @args|
$ll := {
|@ $args|
"List files non-recursively"
" sort_by can be one of ^user, ^size, ^modified, ^type or ^file.
"
Example:
ll .. sort_by=^modified"
find recursive=false @args | sort sort_by
ll .."
find recursive=$false @ $args | sort file
}

34
src/lang/argument.rs
View File

@ -1,9 +1,14 @@
use crate::lang::errors::{argument_error_legacy, error, CrushResult, argument_error};
/**
Code for managing arguments passed in to commands
*/
use crate::lang::errors::{argument_error, argument_error_legacy, CrushResult, error};
use crate::lang::execution_context::CompileContext;
use crate::lang::value::Value;
use crate::lang::value::ValueDefinition;
use std::collections::HashSet;
use crate::lang::ast::{TrackedString, Location};
use std::fmt::{Display, Formatter};
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::ast::location::Location;
#[derive(Debug, Clone)]
pub enum ArgumentType {
@ -112,35 +117,35 @@ impl Argument {
}
}
pub trait ArgumentVecCompiler {
fn compile(&self, context: &mut CompileContext) -> CrushResult<(Vec<Argument>, Option<Value>)>;
pub trait ArgumentEvaluator {
fn eval(&self, context: &mut CompileContext) -> CrushResult<(Vec<Argument>, Option<Value>)>;
}
impl ArgumentVecCompiler for Vec<ArgumentDefinition> {
fn compile(&self, context: &mut CompileContext) -> CrushResult<(Vec<Argument>, Option<Value>)> {
impl ArgumentEvaluator for Vec<ArgumentDefinition> {
fn eval(&self, context: &mut CompileContext) -> CrushResult<(Vec<Argument>, Option<Value>)> {
let mut this = None;
let mut res = Vec::new();
for a in self {
if a.argument_type.is_this() {
this = Some(a.value.compile_bound(context)?);
this = Some(a.value.eval_and_bind(context)?);
} else {
match &a.argument_type {
ArgumentType::Some(name) => {
res.push(Argument::named(
&name.string,
a.value.compile_bound(context)?,
a.value.eval_and_bind(context)?,
a.location,
))
}
ArgumentType::None => {
res.push(Argument::unnamed(
a.value.compile_bound(context)?,
a.value.eval_and_bind(context)?,
a.location,
))
}
ArgumentType::ArgumentList => match a.value.compile_bound(context)? {
ArgumentType::ArgumentList => match a.value.eval_and_bind(context)? {
Value::List(l) => {
let mut copy = l.dump();
for v in copy.drain(..) {
@ -153,7 +158,7 @@ impl ArgumentVecCompiler for Vec<ArgumentDefinition> {
_ => return argument_error_legacy("Argument list must be of type list"),
},
ArgumentType::ArgumentDict => match a.value.compile_bound(context)? {
ArgumentType::ArgumentDict => match a.value.eval_and_bind(context)? {
Value::Dict(d) => {
let mut copy = d.elements();
for (key, value) in copy.drain(..) {
@ -177,6 +182,13 @@ impl ArgumentVecCompiler for Vec<ArgumentDefinition> {
}
}
impl Display for ArgumentDefinition {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
self.value.fmt(f)
}
}
pub fn column_names(arguments: &Vec<Argument>) -> Vec<String> {
let mut taken = HashSet::new();
taken.insert("_".to_string());

28
src/lang/ast/location.rs Normal file
View File

@ -0,0 +1,28 @@
use std::cmp::{max, min};
#[derive(Clone, Debug, Copy, PartialEq, Eq)]
pub struct Location {
pub start: usize,
pub end: usize,
}
impl Location {
pub fn new(start: usize, end: usize) -> Location {
Location { start, end }
}
pub fn union(&self, other: Location) -> Location {
Location {
start: min(self.start, other.start),
end: max(self.end, other.end),
}
}
pub fn contains(&self, cursor: usize) -> bool {
cursor >= self.start && cursor <= self.end
}
pub fn len(&self) -> usize {
self.end - self.start
}
}

220
src/lang/ast.rs → src/lang/ast/mod.rs
View File

@ -1,7 +1,7 @@
use crate::lang::argument::ArgumentDefinition;
use crate::lang::command::{Command, Parameter};
use crate::lang::command_invocation::CommandInvocation;
use crate::lang::errors::{error, to_crush_error, CrushResult};
use crate::lang::errors::{CrushResult, error, to_crush_error};
use crate::lang::job::Job;
use crate::lang::data::scope::Scope;
use crate::lang::value::{Value, ValueDefinition, ValueType};
@ -10,9 +10,36 @@ use regex::Regex;
use std::ops::Deref;
use std::path::PathBuf;
use std::fmt::{Display, Formatter};
use std::cmp::{min, max};
use std::cmp::{max, min};
use location::Location;
use tracked_string::TrackedString;
use crate::util::escape::unescape;
pub mod location;
pub mod tracked_string;
/**
A type representing a node in the abstract syntax tree that is the output of parsing a Crush script.
*/
#[derive(Clone, Debug)]
pub enum Node {
Assignment(Box<Node>, String, Box<Node>),
Unary(TrackedString, Box<Node>),
Glob(TrackedString),
Identifier(TrackedString),
Regex(TrackedString),
Field(TrackedString),
String(TrackedString),
File(TrackedString, bool), // true if filename is quoted
Integer(TrackedString),
Float(TrackedString),
GetItem(Box<Node>, Box<Node>),
GetAttr(Box<Node>, TrackedString),
Path(Box<Node>, TrackedString),
Substitution(JobNode),
Closure(Option<Vec<ParameterNode>>, JobListNode),
}
#[derive(Clone, Debug)]
pub struct JobListNode {
pub jobs: Vec<JobNode>,
@ -166,7 +193,7 @@ impl CommandNode {
error("Stray arguments")
}
} else {
let cmd = self.expressions[0].generate_argument(env)?;
let cmd = self.expressions[0].generate_command(env)?;
let arguments = self.expressions[1..]
.iter()
.map(|e| e.generate_argument(env))
@ -176,99 +203,6 @@ impl CommandNode {
}
}
#[derive(Clone, Debug)]
pub struct TrackedString {
pub string: String,
pub location: Location,
}
impl TrackedString {
pub fn from(string: &str, location: Location) -> TrackedString {
TrackedString {
string: string.to_string(),
location,
}
}
pub fn literal(start: usize, string: &str, end: usize) -> TrackedString {
TrackedString {
string: string.to_string(),
location: Location::new(start, end),
}
}
pub fn prefix(&self, pos: usize) -> TrackedString {
if !self.location.contains(pos) {
if self.location.start > pos {
TrackedString {
string: "".to_string(),
location: Location::new(self.location.start, self.location.start),
}
} else {
self.clone()
}
} else {
let len = pos - self.location.start;
TrackedString {
string: self.string[0..len].to_string(),
location: Location::new(self.location.start, self.location.start + len),
}
}
}
}
impl Display for TrackedString {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.string)
}
}
#[derive(Clone, Debug, Copy, PartialEq, Eq)]
pub struct Location {
pub start: usize,
pub end: usize,
}
impl Location {
pub fn new(start: usize, end: usize) -> Location {
Location { start, end }
}
pub fn union(&self, other: Location) -> Location {
Location {
start: min(self.start, other.start),
end: max(self.end, other.end),
}
}
pub fn contains(&self, cursor: usize) -> bool {
cursor >= self.start && cursor <= self.end
}
pub fn len(&self) -> usize {
self.end - self.start
}
}
#[derive(Clone, Debug)]
pub enum Node {
Assignment(Box<Node>, String, Box<Node>),
Unary(TrackedString, Box<Node>),
Glob(TrackedString),
Label(TrackedString),
Regex(TrackedString),
Field(TrackedString),
String(TrackedString),
File(TrackedString, bool),
Integer(TrackedString),
Float(TrackedString),
GetItem(Box<Node>, Box<Node>),
GetAttr(Box<Node>, TrackedString),
Path(Box<Node>, TrackedString),
Substitution(JobNode),
Closure(Option<Vec<ParameterNode>>, JobListNode),
}
fn propose_name(name: &TrackedString, v: ValueDefinition) -> ValueDefinition {
match v {
ValueDefinition::ClosureDefinition(_, p, j, l) =>
@ -280,7 +214,7 @@ fn propose_name(name: &TrackedString, v: ValueDefinition) -> ValueDefinition {
impl Node {
pub fn prefix(&self, pos: usize) -> CrushResult<Node> {
match self {
Node::Label(s) => Ok(Node::Label(s.prefix(pos))),
Node::Identifier(s) => Ok(Node::Identifier(s.prefix(pos))),
_ => Ok(self.clone()),
}
}
@ -289,7 +223,7 @@ impl Node {
use Node::*;
match self {
Glob(s) | Label(s) | Field(s) |
Glob(s) | Identifier(s) | Field(s) |
String(s) | Integer(s) | Float(s) |
Regex(s) | File(s, _) =>
s.location,
@ -311,16 +245,44 @@ impl Node {
}
}
pub fn generate_command(&self, env: &Scope) -> CrushResult<ArgumentDefinition> {
self.generate(env, true)
}
pub fn generate_argument(&self, env: &Scope) -> CrushResult<ArgumentDefinition> {
self.generate(env, false)
}
pub fn type_name(&self) -> &str {
match self {
Node::Assignment(_, _, _) => "assignment",
Node::Unary(_, _) => "unary operator",
Node::Glob(_) => "glob",
Node::Identifier(_) => "identifier",
Node::Regex(_) => "regular expression literal",
Node::Field(_) => "field",
Node::String(_) => "quoted string literal",
Node::File(_, _) => "file literal",
Node::Integer(_) => "integer literal",
Node::Float(_) => "floating point number literal",
Node::GetItem(_, _) => "subscript",
Node::GetAttr(_, _) => "member access",
Node::Path(_, _) => "path segment",
Node::Substitution(_) => "command substitution",
Node::Closure(_, _) => "closure",
}
}
pub fn generate(&self, env: &Scope, is_command: bool) -> CrushResult<ArgumentDefinition> {
Ok(ArgumentDefinition::unnamed(match self {
Node::Assignment(target, op, value) => match op.deref() {
"=" => {
return match target.as_ref() {
Node::Label(t) => Ok(ArgumentDefinition::named(
Node::Field(t) => Ok(ArgumentDefinition::named(
t.deref(),
propose_name(&t, value.generate_argument(env)?.unnamed_value()?),
)),
_ => error("Invalid left side in named argument"),
_ => error(format!("Invalid left side in named argument. Expected a field, got a {}", target.type_name())),
};
}
_ => return error("Invalid assignment operator"),
@ -346,7 +308,7 @@ impl Node {
}
_ => return error("Unknown operator"),
},
Node::Label(l) => ValueDefinition::Label(l.clone()),
Node::Identifier(l) => ValueDefinition::Identifier(l.clone()),
Node::Regex(l) => ValueDefinition::Value(
Value::Regex(
l.string.clone(),
@ -366,21 +328,19 @@ impl Node {
s.string.replace("_", "").parse::<f64>()
)?),
s.location),
Node::GetAttr(node, label) => {
let parent = node.generate_argument(env)?;
match parent.unnamed_value()? {
ValueDefinition::Value(Value::Field(mut f), location) => {
f.push(label.string.clone());
ValueDefinition::Value(Value::Field(f), location)
}
value => ValueDefinition::GetAttr(Box::new(value), label.clone()),
}
}
Node::Path(node, label) => ValueDefinition::Path(
Node::GetAttr(node, identifier) =>
ValueDefinition::GetAttr(Box::new(node.generate(env, is_command)?.unnamed_value()?), identifier.clone()),
Node::Path(node, identifier) => ValueDefinition::Path(
Box::new(node.generate_argument(env)?.unnamed_value()?),
label.clone(),
identifier.clone(),
),
Node::Field(f) => ValueDefinition::Value(Value::Field(vec![f.string[1..].to_string()]), f.location),
Node::Field(f) =>
if is_command {
ValueDefinition::Identifier(f.clone())
} else {
ValueDefinition::Value(Value::Symbol(f.string.to_string()), f.location)
},
Node::Substitution(s) => ValueDefinition::JobDefinition(s.generate(env)?),
Node::Closure(s, c) => {
let param = s.as_ref().map(|v| {
@ -413,7 +373,7 @@ impl Node {
) -> CrushResult<Option<CommandInvocation>> {
match op.deref() {
"=" => match target.as_ref() {
Node::Label(t) => Node::function_invocation(
Node::Identifier(t) => Node::function_invocation(
env.global_static_cmd(vec!["global", "var", "set"])?,
t.location,
vec![ArgumentDefinition::named(
@ -429,6 +389,7 @@ impl Node {
ArgumentDefinition::unnamed(value.generate_argument(env)?.unnamed_value()?),
],
env,
true,
),
Node::GetAttr(container, attr) => container.method_invocation(
@ -441,12 +402,13 @@ impl Node {
ArgumentDefinition::unnamed(value.generate_argument(env)?.unnamed_value()?),
],
env,
true,
),
_ => error("Invalid left side in assignment"),
},
":=" => match target.as_ref() {
Node::Label(t) => Node::function_invocation(
Node::Identifier(t) => Node::function_invocation(
env.global_static_cmd(vec!["global", "var", "let"])?,
t.location,
vec![ArgumentDefinition::named(
@ -467,7 +429,7 @@ impl Node {
}
Node::GetItem(val, key) => {
val.method_invocation(&TrackedString::from("__getitem__", key.location()), vec![key.generate_argument(env)?], env)
val.method_invocation(&TrackedString::from("__getitem__", key.location()), vec![key.generate_argument(env)?], env, true)
}
Node::Unary(op, _) => match op.string.as_ref() {
@ -476,7 +438,7 @@ impl Node {
},
Node::Glob(_)
| Node::Label(_)
| Node::Identifier(_)
| Node::Regex(_)
| Node::Field(_)
| Node::String(_)
@ -506,24 +468,31 @@ impl Node {
name: &TrackedString,
arguments: Vec<ArgumentDefinition>,
env: &Scope,
as_command: bool,
) -> CrushResult<Option<CommandInvocation>> {
Ok(Some(CommandInvocation::new(
ValueDefinition::GetAttr(
Box::from(self.generate_argument(env)?.unnamed_value()?),
Box::from(self.generate(env, as_command)?.unnamed_value()?),
name.clone(),
),
arguments,
)))
}
pub fn parse_label_or_wildcard(s: &TrackedString) -> Box<Node> {
pub fn parse_string_or_wildcard(s: &TrackedString) -> Box<Node> {
if s.string.contains('%') || s.string.contains('?') {
Box::from(Node::Glob(s.clone()))
} else if s.string.contains('/') || s.string.contains('.') {
Box::from(Node::File(s.clone(), false))
} else {
Box::from(Node::Label(s.clone()))
Box::from(Node::Field(s.clone()))
}
}
pub fn parse_identifier(s: &TrackedString) -> Box<Node> {
Box::from(Node::Identifier(TrackedString::from(&s.string[1..], s.location)))
}
pub fn parse_file_or_wildcard(s: &TrackedString) -> Box<Node> {
if s.string.contains('%') || s.string.contains('?') {
Box::from(Node::Glob(s.clone()))
@ -534,7 +503,7 @@ impl Node {
}
fn path(parts: &[&str], location: Location) -> Node {
let mut res = Node::Label(TrackedString::from(parts[0], location));
let mut res = Node::Identifier(TrackedString::from(parts[0], location));
for part in &parts[1..] {
res = Node::Path(Box::from(res), TrackedString::from(part, location));
}
@ -542,7 +511,7 @@ fn path(parts: &[&str], location: Location) -> Node {
}
fn attr(parts: &[&str], location: Location) -> Node {
let mut res = Node::Label(TrackedString::from(parts[0], location));
let mut res = Node::Identifier(TrackedString::from(parts[0], location));
for part in &parts[1..] {
res = Node::GetAttr(Box::from(res), TrackedString::from(part, location));
}
@ -584,7 +553,7 @@ fn expand_user(s: &str, location: Location) -> Box<Node> {
simple_substitution(
vec![
attr(&vec!["global", "user", "find"], location),
Node::String(TrackedString::from(&format!("\"{}\"", &s[1..]), location))
Node::String(TrackedString::from(&format!("\"{}\"", &s[1..]), location)),
],
location,
),
@ -660,7 +629,8 @@ pub enum TokenType {
FactorOperator,
TermOperator,
QuotedString,
LabelOrWildcard,
StringOrWildcard,
Identifier,
Flag,
Field,
QuotedFile,

49
src/lang/ast/tracked_string.rs Normal file
View File

@ -0,0 +1,49 @@
use std::fmt::{Display, Formatter};
use crate::lang::ast::location::Location;
#[derive(Clone, Debug)]
pub struct TrackedString {
pub string: String,
pub location: Location,
}
impl TrackedString {
pub fn from(string: &str, location: Location) -> TrackedString {
TrackedString {
string: string.to_string(),
location,
}
}
pub fn literal(start: usize, string: &str, end: usize) -> TrackedString {
TrackedString {
string: string.to_string(),
location: Location::new(start, end),
}
}
pub fn prefix(&self, pos: usize) -> TrackedString {
if !self.location.contains(pos) {
if self.location.start > pos {
TrackedString {
string: "".to_string(),
location: Location::new(self.location.start, self.location.start),
}
} else {
self.clone()
}
} else {
let len = pos - self.location.start;
TrackedString {
string: self.string[0..len].to_string(),
location: Location::new(self.location.start, self.location.start + len),
}
}
}
}
impl Display for TrackedString {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.string)
}
}

181
src/lang/command/closure.rs
View File

@ -1,9 +1,9 @@
use crate::lang::argument::{Argument, ArgumentDefinition, ArgumentType};
use crate::lang::command::{BoundCommand, Command, CrushCommand, OutputType, Parameter, ArgumentDescription};
use crate::lang::command::{ArgumentDescription, BoundCommand, Command, CrushCommand, OutputType, Parameter};
use crate::lang::command_invocation::CommandInvocation;
use crate::lang::data::dict::Dict;
use crate::lang::errors::{argument_error_legacy, error, mandate, CrushResult};
use crate::lang::execution_context::{CompileContext, CommandContext, JobContext};
use crate::lang::errors::{argument_error, argument_error_legacy, CrushResult, error, mandate};
use crate::lang::execution_context::{CommandContext, CompileContext, JobContext};
use crate::lang::help::Help;
use crate::lang::job::Job;
use crate::lang::data::list::List;
@ -16,7 +16,8 @@ use crate::lang::pipe::{black_hole, empty_channel};
use crate::lang::value::{Value, ValueDefinition, ValueType};
use std::collections::HashMap;
use std::fmt::Display;
use crate::lang::ast::{TrackedString, Location};
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::ast::location::Location;
pub struct Closure {
name: Option<TrackedString>,
@ -29,7 +30,7 @@ pub struct Closure {
}
impl CrushCommand for Closure {
fn invoke(&self, context: CommandContext) -> CrushResult<()> {
fn eval(&self, context: CommandContext) -> CrushResult<()> {
let job_definitions = self.job_definitions.clone();
let parent_env = self.env.clone();
let env = parent_env.create_child(&context.scope, false);
@ -58,7 +59,7 @@ impl CrushCommand for Closure {
black_hole()
};
let job = job_definition.invoke(JobContext::new(
let job = job_definition.eval(JobContext::new(
input,
output,
env.clone(),
@ -75,7 +76,7 @@ impl CrushCommand for Closure {
Ok(())
}
fn can_block(&self, _arg: &[ArgumentDefinition], _context: &mut CompileContext) -> bool {
fn might_block(&self, _arg: &[ArgumentDefinition], _context: &mut CompileContext) -> bool {
true
}
@ -114,7 +115,7 @@ impl CrushCommand for Closure {
})
}
fn output(&self, _input: &OutputType) -> Option<&ValueType> {
fn output_type(&self, _input: &OutputType) -> Option<&ValueType> {
None
}
@ -201,7 +202,6 @@ impl<'a> ClosureSerializer<'a> {
model::parameter::Parameter::Named(n.serialize(self.elements, self.state)? as u64),
Parameter::Parameter(n, t, d) => {
model::parameter::Parameter::Normal(model::NormalParameter {
name: n.serialize(self.elements, self.state)? as u64,
r#type: Some(self.value_definition(t)?),
@ -303,7 +303,7 @@ impl<'a> ClosureSerializer<'a> {
model::value_definition::ValueDefinition::Job(self.job(j)?)
}
ValueDefinition::Label(l) => {
ValueDefinition::Identifier(l) => {
model::value_definition::ValueDefinition::Label(
l.serialize(self.elements, self.state)? as u64)
}
@ -485,7 +485,7 @@ impl<'a> ClosureDeserializer<'a> {
))
}
model::value_definition::ValueDefinition::Label(s) => {
ValueDefinition::Label(TrackedString::deserialize(*s as usize, self.elements, self.state)?)
ValueDefinition::Identifier(TrackedString::deserialize(*s as usize, self.elements, self.state)?)
}
model::value_definition::ValueDefinition::GetAttr(a) => ValueDefinition::GetAttr(
Box::from(self.value_definition(mandate(
@ -531,13 +531,14 @@ impl Help for Closure {
}
}
/** Extracts the help message from a closure definition */
fn extract_help(jobs: &mut Vec<Job>) -> String {
if jobs.is_empty() {
return "".to_string();
}
let j = &jobs[0];
match j.as_string() {
match j.extract_help_message() {
Some(help) => {
if jobs.len() > 1 {
jobs.remove(0);
@ -570,80 +571,98 @@ impl Closure {
}
}
fn push_arguments_to_env_with_signature(
signature: &Vec<Parameter>,
mut arguments: Vec<Argument>,
context: &mut CompileContext,
) -> CrushResult<()> {
let mut named = HashMap::new();
let mut unnamed = Vec::new();
for arg in arguments.drain(..) {
match arg.argument_type {
Some(name) => {
named.insert(name.clone(), arg.value);
}
None => unnamed.push(arg.value),
};
}
let mut unnamed_name = None;
let mut named_name = None;
for param in signature {
match param {
Parameter::Parameter(name, value_type, default) => {
if let Value::Type(value_type) = value_type.eval_and_bind(context)? {
if named.contains_key(&name.string) {
let value = named.remove(&name.string).unwrap();
if !value_type.is(&value) {
return argument_error(
format!(
"Wrong parameter type {}, expected {}",
value.value_type(), value_type),
name.location);
}
context.env.redeclare(&name.string, value)?;
} else if !unnamed.is_empty() {
context.env.redeclare(&name.string, unnamed.remove(0))?;
} else if let Some(default) = default {
let env = context.env.clone();
env.redeclare(&name.string, default.eval_and_bind(context)?)?;
} else {
return argument_error(
format!(
"Missing variable {}. Options are {}!!!",
name.string,
named.keys().map(|a|{a.to_string()}).collect::<Vec<String>>().join(", ")),
name.location);
}
} else {
return argument_error_legacy("Not a type");
}
}
Parameter::Named(name) => {
if named_name.is_some() {
return argument_error_legacy("Multiple named argument maps specified");
}
named_name = Some(name);
}
Parameter::Unnamed(name) => {
if unnamed_name.is_some() {
return argument_error_legacy("Multiple named argument maps specified");
}
unnamed_name = Some(name);
}
}
}
if let Some(unnamed_name) = unnamed_name {
context.env.redeclare(
unnamed_name.string.as_ref(),
Value::List(List::new(ValueType::Any, unnamed)),
)?;
} else if !unnamed.is_empty() {
return argument_error_legacy("No target for unnamed arguments");
}
if let Some(named_name) = named_name {
let d = Dict::new(ValueType::String, ValueType::Any);
for (k, v) in named {
d.insert(Value::string(&k), v)?;
}
context.env.redeclare(named_name.string.as_ref(), Value::Dict(d))?;
} else if !named.is_empty() {
return argument_error_legacy("No target for extra named arguments");
}
Ok(())
}
fn push_arguments_to_env(
signature: &Option<Vec<Parameter>>,
mut arguments: Vec<Argument>,
context: &mut CompileContext,
) -> CrushResult<()> {
if let Some(signature) = signature {
let mut named = HashMap::new();
let mut unnamed = Vec::new();
for arg in arguments.drain(..) {
match arg.argument_type {
Some(name) => {
named.insert(name.clone(), arg.value);
}
None => unnamed.push(arg.value),
};
}
let mut unnamed_name = None;
let mut named_name = None;
for param in signature {
match param {
Parameter::Parameter(name, value_type, default) => {
if let Value::Type(value_type) = value_type.compile_bound(context)? {
if named.contains_key(&name.string) {
let value = named.remove(&name.string).unwrap();
if !value_type.is(&value) {
return argument_error_legacy("Wrong parameter type");
}
context.env.redeclare(&name.string, value)?;
} else if !unnamed.is_empty() {
context.env.redeclare(&name.string, unnamed.remove(0))?;
} else if let Some(default) = default {
let env = context.env.clone();
env.redeclare(&name.string, default.compile_bound(context)?)?;
} else {
return argument_error_legacy("Missing variable!!!");
}
} else {
return argument_error_legacy("Not a type");
}
}
Parameter::Named(name) => {
if named_name.is_some() {
return argument_error_legacy("Multiple named argument maps specified");
}
named_name = Some(name);
}
Parameter::Unnamed(name) => {
if unnamed_name.is_some() {
return argument_error_legacy("Multiple named argument maps specified");
}
unnamed_name = Some(name);
}
}
}
if let Some(unnamed_name) = unnamed_name {
context.env.redeclare(
unnamed_name.string.as_ref(),
Value::List(List::new(ValueType::Any, unnamed)),
)?;
} else if !unnamed.is_empty() {
return argument_error_legacy("No target for unnamed arguments");
}
if let Some(named_name) = named_name {
let d = Dict::new(ValueType::String, ValueType::Any);
for (k, v) in named {
d.insert(Value::string(&k), v)?;
}
context.env.redeclare(named_name.string.as_ref(), Value::Dict(d))?;
} else if !named.is_empty() {
return argument_error_legacy("No target for extra named arguments");
}
Self::push_arguments_to_env_with_signature(signature, arguments, context)
} else {
for arg in arguments.drain(..) {
match arg.argument_type {
@ -655,8 +674,8 @@ impl Closure {
}
}
}
Ok(())
}
Ok(())
}
pub fn deserialize(

38
src/lang/command/mod.rs
View File

@ -13,7 +13,7 @@ use crate::lang::value::{Value, ValueDefinition, ValueType};
use closure::Closure;
use ordered_map::OrderedMap;
use std::fmt::{Formatter, Display};
use crate::lang::ast::TrackedString;
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::completion::Completion;
use crate::lang::completion::parse::PartialCommandResult;
@ -62,8 +62,8 @@ pub struct ArgumentDescription {
}
pub trait CrushCommand: Help {
fn invoke(&self, context: CommandContext) -> CrushResult<()>;
fn can_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool;
fn eval(&self, context: CommandContext) -> CrushResult<()>;
fn might_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool;
fn name(&self) -> &str;
fn copy(&self) -> Command;
fn help(&self) -> &dyn Help;
@ -73,7 +73,7 @@ pub trait CrushCommand: Help {
state: &mut SerializationState,
) -> CrushResult<usize>;
fn bind(&self, this: Value) -> Command;
fn output<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType>;
fn output_type<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType>;
fn arguments(&self) -> &Vec<ArgumentDescription>;
}
@ -220,12 +220,12 @@ impl dyn CrushCommand {
}
impl CrushCommand for SimpleCommand {
fn invoke(&self, context: CommandContext) -> CrushResult<()> {
fn eval(&self, context: CommandContext) -> CrushResult<()> {
let c = self.call;
c(context)
}
fn can_block(&self, _arg: &[ArgumentDefinition], _context: &mut CompileContext) -> bool {
fn might_block(&self, _arg: &[ArgumentDefinition], _context: &mut CompileContext) -> bool {
self.can_block
}
@ -270,7 +270,7 @@ impl CrushCommand for SimpleCommand {
})
}
fn output<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType> {
fn output_type<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType> {
self.output.calculate(input)
}
@ -315,7 +315,7 @@ impl std::fmt::Debug for SimpleCommand {
}
impl CrushCommand for ConditionCommand {
fn invoke(&self, context: CommandContext) -> CrushResult<()> {
fn eval(&self, context: CommandContext) -> CrushResult<()> {
let c = self.call;
c(context)?;
Ok(())
@ -325,10 +325,10 @@ impl CrushCommand for ConditionCommand {
"conditional command"
}
fn can_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
fn might_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
arguments
.iter()
.any(|arg| arg.value.can_block(arguments, context))
.any(|arg| arg.value.can_block(context))
}
fn copy(&self) -> Command {
@ -365,7 +365,7 @@ impl CrushCommand for ConditionCommand {
})
}
fn output(&self, _input: &OutputType) -> Option<&ValueType> {
fn output_type(&self, _input: &OutputType) -> Option<&ValueType> {
None
}
@ -388,13 +388,13 @@ impl Help for ConditionCommand {
}
}
impl std::cmp::PartialEq for ConditionCommand {
impl PartialEq for ConditionCommand {
fn eq(&self, _other: &ConditionCommand) -> bool {
false
}
}
impl std::cmp::Eq for ConditionCommand {}
impl Eq for ConditionCommand {}
#[derive(Clone)]
pub enum Parameter {
@ -434,13 +434,13 @@ pub struct BoundCommand {
}
impl CrushCommand for BoundCommand {
fn invoke(&self, mut context: CommandContext) -> CrushResult<()> {
fn eval(&self, mut context: CommandContext) -> CrushResult<()> {
context.this = Some(self.this.clone());
self.command.invoke(context)
self.command.eval(context)
}
fn can_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
self.command.can_block(arguments, context)
fn might_block(&self, arguments: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
self.command.might_block(arguments, context)
}
fn name(&self) -> &str {
@ -482,8 +482,8 @@ impl CrushCommand for BoundCommand {
})
}
fn output<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType> {
self.command.output(input)
fn output_type<'a>(&'a self, input: &'a OutputType) -> Option<&'a ValueType> {
self.command.output_type(input)
}
fn arguments(&self) -> &Vec<ArgumentDescription> {

540
src/lang/command_invocation.rs
View File

@ -1,15 +1,16 @@
use crate::lang::errors::{error, CrushResult, CrushErrorType};
use crate::lang::errors::{CrushResult, error};
use crate::lang::execution_context::{CompileContext, JobContext};
use crate::lang::data::scope::Scope;
use crate::lang::{argument::ArgumentDefinition, argument::ArgumentVecCompiler, value::Value};
use crate::lang::{argument::ArgumentDefinition, argument::ArgumentEvaluator, value::Value};
use crate::lang::command::Command;
use crate::lang::execution_context::CommandContext;
use crate::lang::value::ValueDefinition;
use std::ops::Deref;
use crate::lang::value::{ValueDefinition, ValueType};
use std::path::PathBuf;
use std::fmt::{Display, Formatter};
use std::thread::ThreadId;
use crate::lang::ast::Location;
use crate::data::r#struct::Struct;
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::ast::location::Location;
#[derive(Clone)]
pub struct CommandInvocation {
@ -17,6 +18,239 @@ pub struct CommandInvocation {
arguments: Vec<ArgumentDefinition>,
}
fn arg_can_block(local_arguments: &Vec<ArgumentDefinition>, context: &mut CompileContext) -> bool {
for arg in local_arguments {
if arg.value.can_block(context) {
return true;
}
}
false
}
impl CommandInvocation {
pub fn new(command: ValueDefinition, arguments: Vec<ArgumentDefinition>) -> CommandInvocation {
CommandInvocation { command, arguments }
}
/** Extracts the help message from a closure definition */
pub fn extract_help_message(&self) -> Option<String> {
if self.arguments.len() != 0 {
return None;
}
match &self.command {
ValueDefinition::Value(Value::String(s), _) => Some(s.to_string()),
_ => None,
}
}
pub fn arguments(&self) -> &[ArgumentDefinition] {
&self.arguments
}
pub fn command(&self) -> &ValueDefinition {
&self.command
}
/**
Evaluates all the arguments into values, and puts them into a CommandContext,
ready to be exacuted by the main command.
*/
fn execution_context(
local_arguments: Vec<ArgumentDefinition>,
mut this: Option<Value>,
job_context: JobContext,
) -> CrushResult<CommandContext> {
let (arguments, arg_this) = local_arguments.eval(&mut CompileContext::from(&job_context))?;
if arg_this.is_some() {
this = arg_this;
}
Ok(job_context.command_context(arguments, this))
}
pub fn can_block(&self, context: &mut CompileContext) -> bool {
if self.command.can_block(context) {
return true;
}
match self.command.eval(context) {
Ok((_, Value::Command(command))) =>
command.might_block(&self.arguments, context) || arg_can_block(&self.arguments, context),
_ => true,
}
}
pub fn eval(&self, context: JobContext) -> CrushResult<Option<ThreadId>> {
eval(&self.command, &self.arguments, context)
}
}
pub fn eval_non_blocking(command: &ValueDefinition, arguments: &Vec<ArgumentDefinition>, context: JobContext) -> CrushResult<Option<ThreadId>> {
match command.eval(&mut CompileContext::from(&context))
{
// Try to find the command in this thread. This may fail if the command is found via a subshell, in which case we need to spawn a thread
Ok((this, value)) => {
eval_internal(this, value, arguments.clone(), context, command.location())
}
Err(err) =>
if let ValueDefinition::Identifier(str) = command {
try_external_command(str, arguments.clone(), context)
} else {
return Err(err);
}
}
}
pub fn eval(command: &ValueDefinition, arguments: &Vec<ArgumentDefinition>, context: JobContext) -> CrushResult<Option<ThreadId>> {
if command.can_block(&mut CompileContext::from(&context)) {
eval_non_blocking(command, arguments, context)
} else {
let command = command.clone();
let arguments = arguments.clone();
let t = context.global_state.threads().clone();
Ok(Some(t.spawn(
&command.to_string(),
move || {
match eval_non_blocking(&command, &arguments, context.clone()) {
Ok(Some(id)) => context.global_state.threads().join_one(
id,
&context.global_state.printer(),
),
Err(e) => context.global_state.printer().crush_error(e),
_ => {}
}
Ok(())
},
)?))
}
}
fn eval_internal(
this: Option<Value>,
value: Value,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
location: Location,
) -> CrushResult<Option<ThreadId>> {
match value {
Value::Command(command) => eval_command(command, this, local_arguments, context),
Value::Type(t) => eval_type(t, local_arguments, context, location),
Value::Struct(s) => eval_struct(s, local_arguments, context, location),
v => eval_other(v, local_arguments, context, location),
}
}
fn eval_other(
value: Value,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
location: Location,
) -> CrushResult<Option<ThreadId>> {
if local_arguments.len() == 0 {
eval_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(value, location))],
context,
)
} else {
error(&format!("{} is not a command.", value))
}
}
fn eval_type(
value_type: ValueType,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
location: Location,
) -> CrushResult<Option<ThreadId>> {
match value_type.fields().get("__call__") {
None => eval_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::Type(value_type),
location,
))],
context,
),
Some(call) => eval_command(
call.as_ref().copy(),
Some(Value::Type(value_type)),
local_arguments,
context,
),
}
}
fn eval_struct(
struct_value: Struct,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
location: Location,
) -> CrushResult<Option<ThreadId>> {
match struct_value.get("__call__") {
Some(Value::Command(call)) => {
eval_command(call, Some(Value::Struct(struct_value)), local_arguments, context)
}
Some(v) => error(
format!(
"__call__ should be a command, was of type {}",
v.value_type().to_string()
)
.as_str(),
),
_ => {
if local_arguments.len() == 0 {
eval_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::Struct(struct_value),
location,
))],
context,
)
} else {
error(
format!(
"Struct must have a member __call__ to be used as a command {}",
struct_value.to_string()
)
.as_str(),
)
}
}
}
}
fn eval_command(
command: Command,
this: Option<Value>,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
) -> CrushResult<Option<ThreadId>> {
if !command.might_block(&local_arguments, &mut CompileContext::from(&context))
&& !arg_can_block(&local_arguments, &mut CompileContext::from(&context))
{
let new_context =
CommandInvocation::execution_context(local_arguments, this, context.clone())?;
context.global_state.printer().handle_error(command.eval(new_context));
Ok(None)
} else {
let t = context.global_state.threads().clone();
let name = command.name().to_string();
Ok(Some(t.spawn(
&name,
move || {
let res = CommandInvocation::execution_context(local_arguments, this, context.clone())?;
command.eval(res)
},
)?))
}
}
fn resolve_external_command(name: &str, env: &Scope) -> CrushResult<Option<PathBuf>> {
if let Some(Value::List(path)) = env.get("cmd_path")? {
let path_vec = path.dump();
@ -35,284 +269,11 @@ fn resolve_external_command(name: &str, env: &Scope) -> CrushResult<Option<PathB
Ok(None)
}
fn arg_can_block(local_arguments: &Vec<ArgumentDefinition>, context: &mut CompileContext) -> bool {
for arg in local_arguments {
if arg.value.can_block(local_arguments, context) {
return true;
}
}
false
}
impl CommandInvocation {
pub fn new(command: ValueDefinition, arguments: Vec<ArgumentDefinition>) -> CommandInvocation {
CommandInvocation { command, arguments }
}
pub fn as_string(&self) -> Option<String> {
if self.arguments.len() != 0 {
return None;
}
match &self.command {
ValueDefinition::Value(Value::String(s), _) => Some(s.to_string()),
_ => None,
}
}
pub fn arguments(&self) -> &[ArgumentDefinition] {
&self.arguments
}
pub fn command(&self) -> &ValueDefinition {
&self.command
}
/*
pub fn spawn_stream(
&self,
env: &Scope,
mut argument_stream: InputStream,
output: ValueSender,
) -> CrushResult<JobJoinHandle> {
let cmd = env.get(&self.name);
match cmd {
Some(Value::Command(command)) => {
let c = command.call;
Ok(handle(build(format_name(&self.name)).spawn(
move || {
loop {
match argument_stream.recv() {
Ok(mut row) => {}
Err(_) => break,
}
}
Ok(())
})))
}
_ => {
error("Can't stream call")
}
}
}
*/
fn execution_context(
local_arguments: Vec<ArgumentDefinition>,
mut this: Option<Value>,
job_context: JobContext,
) -> CrushResult<CommandContext> {
let (arguments, arg_this) = local_arguments.compile(&mut CompileContext::from(&job_context))?;
if arg_this.is_some() {
this = arg_this;
}
Ok(job_context.command_context(arguments, this))
}
pub fn can_block(&self, arg: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
let cmd = self.command.compile_internal(context, false);
match cmd {
Ok((_, Value::Command(command))) => {
command.can_block(arg, context) || arg_can_block(&self.arguments, context)
}
_ => true,
}
}
pub fn invoke(&self, context: JobContext) -> CrushResult<Option<ThreadId>> {
match self
.command
.compile_internal(&mut CompileContext::from(&context), false)
{
Ok((this, value)) => invoke_value(this, value, self.arguments.clone(), context, self.command.location()),
Err(err) => {
if err.is(CrushErrorType::BlockError) {
let cmd = self.command.clone();
let arguments = self.arguments.clone();
let t = context.global_state.threads().clone();
let location = self.command.location();
Ok(Some(t.spawn(
&self.command.to_string(),
move || {
match cmd.clone().compile_unbound(&mut CompileContext::from(&context)) {
Ok((this, value)) => context.global_state.printer().handle_error(invoke_value(
this,
value,
arguments,
context.clone(),
location,
)),
_ => context.global_state.printer().handle_error(try_external_command(
cmd,
arguments,
context.clone(),
)),
}
Ok(())
},
)?))
} else {
try_external_command(self.command.clone(), self.arguments.clone(), context)
}
}
}
}
}
fn invoke_value(
this: Option<Value>,
value: Value,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
location: Location,
) -> CrushResult<Option<ThreadId>> {
match value {
Value::Command(command) => invoke_command(command, this, local_arguments, context),
Value::File(f) => {
if local_arguments.len() == 0 {
let meta = f.metadata();
if meta.is_ok() && meta.unwrap().is_dir() {
invoke_command(
context
.scope
.global_static_cmd(vec!["global", "fs", "cd"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::File(f),
location,
))],
context,
)
} else {
invoke_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::File(f),
location,
))],
context,
)
}
} else {
error(
format!(
"Not a command {}",
f.to_str().unwrap_or("<invalid filename>")
)
.as_str(),
)
}
}
Value::Type(t) => match t.fields().get("__call__") {
None => invoke_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::Type(t),
location,
))],
context,
),
Some(call) => invoke_command(
call.as_ref().copy(),
Some(Value::Type(t)),
local_arguments,
context,
),
},
Value::Struct(s) => match s.get("__call__") {
Some(Value::Command(call)) => {
invoke_command(call, Some(Value::Struct(s)), local_arguments, context)
}
Some(v) => error(
format!(
"__call__ should be a command, was of type {}",
v.value_type().to_string()
)
.as_str(),
),
_ => {
if local_arguments.len() == 0 {
invoke_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(
Value::Struct(s),
location,
))],
context,
)
} else {
error(
format!(
"Struct must have a member __call__ to be used as a command {}",
s.to_string()
)
.as_str(),
)
}
}
},
_ => {
if local_arguments.len() == 0 {
invoke_command(
context.scope.global_static_cmd(vec!["global", "io", "val"])?,
None,
vec![ArgumentDefinition::unnamed(ValueDefinition::Value(value, location))],
context,
)
} else {
error(&format!("Not a command {}", value))
}
}
}
}
fn invoke_command(
action: Command,
this: Option<Value>,
local_arguments: Vec<ArgumentDefinition>,
context: JobContext,
) -> CrushResult<Option<ThreadId>> {
if !action.can_block(&local_arguments, &mut CompileContext::from(&context))
&& !arg_can_block(&local_arguments, &mut CompileContext::from(&context))
{
let new_context =
CommandInvocation::execution_context(local_arguments, this, context.clone())?;
context.global_state.printer().handle_error(action.invoke(new_context));
Ok(None)
} else {
let t = context.global_state.threads().clone();
let name = action.name().to_string();
Ok(Some(t.spawn(
&name,
move || {
let res = CommandInvocation::execution_context(local_arguments, this, context.clone())?;
action.invoke(res)
},
)?))
}
}
fn try_external_command(
def: ValueDefinition,
cmd: &TrackedString,
mut arguments: Vec<ArgumentDefinition>,
context: JobContext,
) -> CrushResult<Option<ThreadId>> {
let def_location = def.location();
let (cmd, sub) = match def {
ValueDefinition::Label(str) => (str, None),
ValueDefinition::GetAttr(parent, sub) => match parent.deref() {
ValueDefinition::Label(str) => (str.clone(), Some(sub)),
_ => return error("Not a command"),
},
_ => return error("Not a command"),
};
match resolve_external_command(&cmd.string, &context.scope)? {
None => error(format!("Unknown command name {}", cmd).as_str()),
Some(path) => {
@ -320,16 +281,6 @@ fn try_external_command(
0,
ArgumentDefinition::unnamed(ValueDefinition::Value(Value::File(path), cmd.location)),
);
if let Some(subcmd) = sub {
arguments.insert(
1,
ArgumentDefinition::unnamed(
ValueDefinition::Value(
Value::string(subcmd.string),
subcmd.location,
)),
);
}
let call = CommandInvocation {
command: ValueDefinition::Value(
Value::Command(
@ -337,17 +288,22 @@ fn try_external_command(
.scope
.global_static_cmd(vec!["global", "control", "cmd"])?,
),
def_location,
cmd.location,
),
arguments,
};
call.invoke(context)
call.eval(context)
}
}
}
impl Display for CommandInvocation {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
self.command.fmt(f)
self.command.fmt(f)?;
for a in &self.arguments {
f.write_str(" ")?;
a.fmt(f)?;
}
Ok(())
}
}

39
src/lang/completion/mod.rs
View File

@ -81,7 +81,7 @@ fn completion_suffix(maybe_scope: CrushResult<Option<Value>>, t: &ValueType) ->
Ok(Some(Value::Scope(_))) => ":",
Ok(Some(Value::Empty())) |
Ok(Some(Value::Bool(_))) |
Ok(Some(Value::Field(_))) |
Ok(Some(Value::Symbol(_))) |
Ok(Some(Value::Command(_))) => " ",
_ => "",
}
@ -281,6 +281,9 @@ fn complete_partial_argument(
LastArgument::Label(label) => {
complete_label(Value::Scope(scope.clone()), &label, &argument_type, cursor, res)?;
}
LastArgument::Field(label) => {
if parse_result.last_argument_name.is_none() {
if let CompletionCommand::Known(cmd) = parse_result.command {
complete_argument_name(cmd.arguments(), &label, cursor, res, false)?;
@ -288,7 +291,7 @@ fn complete_partial_argument(
}
}
LastArgument::Field(parent, field) => {
LastArgument::Member(parent, field) => {
complete_label(parent, &field, &argument_type, cursor, res)?;
}
@ -320,6 +323,10 @@ pub fn complete(
complete_label(Value::Scope(scope.clone()), &label, &ValueType::Any, cursor, &mut res)?;
}
ParseResult::PartialField(label) => {
complete_label(Value::Scope(scope.clone()), &label, &ValueType::Any, cursor, &mut res)?;
}
ParseResult::PartialMember(parent, label) => {
complete_label(parent, &label, &ValueType::Any, cursor, &mut res)?;
}
@ -642,8 +649,8 @@ mod tests {
#[test]
fn complete_namespaced_argument() {
let line = "xxx abcd:bc";
let cursor = 11;
let line = "xxx $abcd:bc";
let cursor = line.len();
let s = Scope::create_root();
s.create_namespace("abcd", "bla", Box::new(|env| {
@ -653,19 +660,19 @@ mod tests {
let completions = complete(line, cursor, &s, &parser(), &empty_lister()).unwrap();
assert_eq!(completions.len(), 1);
assert_eq!(&completions[0].complete(line), "xxx abcd:bcde ");
assert_eq!(&completions[0].complete(line), "xxx $abcd:bcde ");
}
#[test]
fn complete_simple_argument() {
let line = "abcd ab";
let cursor = 7;
let line = "abcd $ab";
let cursor = line.len();
let s = Scope::create_root();
s.declare("abcd", Value::Empty()).unwrap();
let completions = complete(line, cursor, &s, &parser(), &empty_lister()).unwrap();
assert_eq!(completions.len(), 1);
assert_eq!(&completions[0].complete(line), "abcd abcd ");
assert_eq!(&completions[0].complete(line), "abcd $abcd ");
}
#[test]
@ -682,26 +689,26 @@ mod tests {
#[test]
fn check_multiple_token() {
let line = "ab cd ef";
let cursor = 5;
let line = "ab $cd ef";
let cursor = 6;
let s = Scope::create_root();
s.declare("cdef", Value::Empty()).unwrap();
let completions = complete(line, cursor, &s, &parser(), &empty_lister()).unwrap();
assert_eq!(completions.len(), 1);
assert_eq!(&completions[0].complete(line), "ab cdef ef");
assert_eq!(&completions[0].complete(line), "ab $cdef ef");
}
#[test]
fn check_named_argument() {
let line = "ab foo=cd";
let cursor = 9;
let line = "ab foo=$cd";
let cursor = 10;
let s = Scope::create_root();
s.declare("cdef", Value::Empty()).unwrap();
let completions = complete(line, cursor, &s, &parser(), &empty_lister()).unwrap();
assert_eq!(completions.len(), 1);
assert_eq!(&completions[0].complete(line), "ab foo=cdef ");
assert_eq!(&completions[0].complete(line), "ab foo=$cdef ");
}
#[test]
@ -718,7 +725,7 @@ mod tests {
#[test]
fn check_completion_type_filtering() {
let line = "my_cmd super_fancy_argument=t";
let line = "my_cmd super_fancy_argument=$t";
let cursor = line.len();
let s = scope_with_function();
@ -726,7 +733,7 @@ mod tests {
s.declare("type", Value::Type(ValueType::Empty)).unwrap();
let completions = complete(line, cursor, &s, &parser(), &empty_lister()).unwrap();
assert_eq!(completions.len(), 1);
assert_eq!(&completions[0].complete(line), "my_cmd super_fancy_argument=type ");
assert_eq!(&completions[0].complete(line), "my_cmd super_fancy_argument=$type ");
}
#[test]

114
src/lang/completion/parse.rs
View File

@ -1,3 +1,4 @@
use std::cmp::min;
use crate::lang::ast::{Node, CommandNode, JobListNode, JobNode};
use crate::lang::errors::{error, CrushResult, mandate, argument_error_legacy, to_crush_error};
use crate::lang::value::{ValueType, Value};
@ -9,6 +10,7 @@ use crate::util::glob::Glob;
use crate::lang::parser::Parser;
use crate::util::escape::unescape;
use std::collections::HashSet;
use std::fmt::{Display, Formatter};
pub enum CompletionCommand {
Unknown,
@ -28,7 +30,8 @@ impl Clone for CompletionCommand {
pub enum LastArgument {
Unknown,
Label(String),
Field(Value, String),
Field(String),
Member(Value, String),
File(String, bool),
QuotedString(String),
Switch(String),
@ -68,9 +71,9 @@ impl PartialCommandResult {
if false && cmd.arguments().len() == 1 {
Some(&cmd.arguments()[0])
} else {
let mut previous_named = HashSet::new();
let mut previous_unnamed = 0usize;
for arg in &self.previous_arguments {
match &arg.name {
Some(name) => {
@ -83,9 +86,9 @@ impl PartialCommandResult {
let mut unnamed_used = 0usize;
for arg in cmd.arguments() {
if arg.unnamed {
return Some(arg)
return Some(arg);
}
if previous_named.contains(&arg.name ) {
if previous_named.contains(&arg.name) {
continue;
} else {
unnamed_used += 1;
@ -115,15 +118,47 @@ impl PartialCommandResult {
pub enum ParseResult {
Nothing,
PartialLabel(String),
PartialField(String),
PartialMember(Value, String),
PartialFile(String, bool),
PartialQuotedString(String),
PartialArgument(PartialCommandResult),
}
impl Display for ParseResult {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
ParseResult::Nothing => f.write_str("nothing"),
ParseResult::PartialLabel(l) => {
f.write_str("label ")?;
f.write_str(l)
}
ParseResult::PartialField(l) => {
f.write_str("field ")?;
f.write_str(l)
}
ParseResult::PartialMember(_p, m) => {
f.write_str("member ")?;
f.write_str(m)
}
ParseResult::PartialFile(p, _q) => {
f.write_str("file ")?;
f.write_str(p)
}
ParseResult::PartialQuotedString(s) => {
f.write_str("string ")?;
f.write_str(s)
}
ParseResult::PartialArgument(_a) => {
f.write_str("command")
}
}
}
}
fn simple_path(node: &Node, cursor: usize) -> CrushResult<String> {
match node {
Node::Label(label) => Ok(label.string.clone()),
Node::Identifier(label) => Ok(label.string.clone()),
Node::Path(p, a) => {
let res = simple_path(p.as_ref(), cursor)?;
Ok(format!("{}/{}", res, &a.string))
@ -192,12 +227,19 @@ fn find_command_in_job_list(ast: JobListNode, cursor: usize) -> CrushResult<Comm
.map(|c| c.clone())), "Nothing to complete")
}
fn fetch_value(node: &Node, scope: &Scope) -> CrushResult<Option<Value>> {
fn fetch_value(node: &Node, scope: &Scope, is_command: bool) -> CrushResult<Option<Value>> {
match node {
Node::Label(l) => scope.get(&l.string),
Node::Identifier(l) => scope.get(&l.string),
Node::Field(l) =>
if is_command {
scope.get(&l.string)
} else {
Ok(None)
},
Node::GetAttr(n, l) =>
match fetch_value(n, scope)? {
match fetch_value(n, scope, is_command)? {
Some(parent) => parent.field(&l.string),
None => Ok(None),
},
@ -225,7 +267,7 @@ fn fetch_value(node: &Node, scope: &Scope) -> CrushResult<Option<Value>> {
}
fn parse_command_node(node: &Node, scope: &Scope) -> CrushResult<CompletionCommand> {
match fetch_value(node, scope)? {
match fetch_value(node, scope, true)? {
Some(Value::Command(command)) => Ok(CompletionCommand::Known(command)),
_ => Ok(CompletionCommand::Unknown),
}
@ -235,18 +277,18 @@ fn parse_previous_argument(arg: &Node) -> PreviousArgument {
match arg {
Node::Assignment(key, op, value) => {
match (key.as_ref(), op.as_str()) {
(Node::Label(name), "=") => {
(Node::Field(name), "=") => {
let inner = parse_previous_argument(value.as_ref());
return PreviousArgument {
name: Some(name.string.clone()),
value: inner.value
}
value: inner.value,
};
}
_ => {}
}
}
_ => {},
_ => {}
}
PreviousArgument {
name: None,
@ -274,13 +316,17 @@ pub fn parse(
let cmd = &cmd.expressions[0];
if cmd.location().contains(cursor) {
match cmd {
Node::Label(label) =>
Node::Identifier(label) =>
Ok(ParseResult::PartialLabel(
label.prefix(cursor).string)),
Node::Field(label) =>
Ok(ParseResult::PartialField(
label.prefix(cursor).string)),
Node::GetAttr(parent, field) =>
Ok(ParseResult::PartialMember(
mandate(fetch_value(parent, scope)?, "Unknown value")?,
mandate(fetch_value(parent, scope, true)?, "Unknown value")?,
field.prefix(cursor).string)),
Node::Path(_, _) =>
@ -296,7 +342,7 @@ pub fn parse(
Node::GetItem(_, _) => { panic!("AAA"); }
_ => error("Can't extract command to complete"),
_ => error(format!("Can't extract command to complete. Unknown node type {}", cmd.type_name())),
}
} else {
Ok(ParseResult::PartialArgument(
@ -321,7 +367,7 @@ pub fn parse(
if name.location().contains(cursor) {
(Box::from(name.prefix(cursor)?), None, true)
} else {
if let Node::Label(name) = name.as_ref() {
if let Node::Identifier(name) = name.as_ref() {
(value.clone(), Some(name.string.clone()), false)
} else {
(value.clone(), None, false)
@ -333,22 +379,24 @@ pub fn parse(
if argument_complete {
match arg.deref() {
Node::Label(l) =>
Node::Field(l) => {
let substring = &l.string[0..min(l.string.len(), cursor - l.location.start)];
Ok(ParseResult::PartialArgument(
PartialCommandResult {
command: c,
previous_arguments,
last_argument: LastArgument::Switch(l.string.clone()),
last_argument: LastArgument::Switch(substring.to_string()),
last_argument_name,
}
)),
))
}
_ => argument_error_legacy("Invalid argument name"),
_ => argument_error_legacy(format!("Invalid argument name {}", arg.type_name()))
}
} else {
if arg.location().contains(cursor) {
match arg.as_ref() {
Node::Label(l) =>
Node::Identifier(l) =>
Ok(ParseResult::PartialArgument(
PartialCommandResult {
command: c,
@ -358,13 +406,23 @@ pub fn parse(
}
)),
Node::Field(l) =>
Ok(ParseResult::PartialArgument(
PartialCommandResult {
command: c,
previous_arguments,
last_argument: LastArgument::Field(l.string.clone()),
last_argument_name,
}
)),
Node::GetAttr(parent, field) =>
Ok(ParseResult::PartialArgument(
PartialCommandResult {
command: c,
previous_arguments,
last_argument: LastArgument::Field(
mandate(fetch_value(parent, scope)?, "unknown value")?,
last_argument: LastArgument::Member(
mandate(fetch_value(parent, scope, false)?, "unknown value")?,
field.prefix(cursor).string),
last_argument_name,
})),
@ -421,7 +479,7 @@ pub fn parse(
#[cfg(test)]
mod tests {
use super::*;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
use crate::lang::parser::lalrparser;
fn ast(s: &str) -> CrushResult<JobListNode> {
@ -451,8 +509,8 @@ mod tests {
#[test]
fn find_command_in_operator() {
let ast = ast("ps | where {^cpu == (max_)}").unwrap();
let cmd = find_command_in_job_list(ast, 25).unwrap();
assert_eq!(cmd.location, Location::new(21, 25))
let ast = ast("ps | where {cpu == (max_)}").unwrap();
let cmd = find_command_in_job_list(ast, 24).unwrap();
assert_eq!(cmd.location, Location::new(20, 24))
}
}

4
src/lang/data/binary.rs
View File

@ -127,9 +127,9 @@ impl dyn BinaryReader {
}
}
pub fn vec(vec: &Vec<u8>) -> Box<dyn BinaryReader + Send + Sync> {
pub fn vec(bytes: &[u8]) -> Box<dyn BinaryReader + Send + Sync> {
Box::from(VecReader {
vec: vec.clone(),
vec: Vec::from(bytes),
offset: 0,
})
}

54
src/lang/data/list.rs
View File

@ -1,6 +1,6 @@
use crate::lang::errors::{argument_error_legacy, error, mandate, CrushResult};
use crate::lang::pipe::CrushStream;
use crate::lang::{data::table::ColumnType, data::table::Row, value::Field, value::Value, value::ValueType};
use crate::lang::pipe::{CrushStream, Stream};
use crate::lang::{data::table::ColumnType, data::table::Row, value::Value, value::ValueType};
use crate::util::identity_arc::Identity;
use chrono::Duration;
use std::cmp::Ordering;
@ -8,6 +8,7 @@ use std::collections::HashSet;
use std::hash::Hasher;
use std::sync::{Arc, Mutex};
use std::fmt::{Display, Formatter};
use crate::lang::value::VecReader;
#[derive(Clone)]
pub struct List {
@ -179,12 +180,18 @@ impl List {
Ok(())
}
pub fn stream(&self) -> Stream {
let mut vec = Vec::new();
self.dump_value(&mut vec);
Box::new(VecReader::new(vec, self.cell_type.clone()))
}
dump_to!(dump_string, String, String, |e: &String| e.to_string());
dump_to!(dump_integer, i128, Integer, |v: &i128| *v);
dump_to!(dump_bool, bool, Bool, |v: &bool| *v);
dump_to!(dump_type, ValueType, Type, |v: &ValueType| v.clone());
dump_to!(dump_float, f64, Float, |v: &f64| *v);
dump_to!(dump_field, Field, Field, |e: &Field| e.clone());
dump_to!(dump_symbol, String, Symbol, |e: &String| e.clone());
}
impl Display for List {
@ -213,7 +220,7 @@ impl std::hash::Hash for List {
}
}
impl std::cmp::PartialEq for List {
impl PartialEq for List {
fn eq(&self, other: &List) -> bool {
let us = self.cells.lock().unwrap().clone();
let them = other.cells.lock().unwrap().clone();
@ -229,7 +236,7 @@ impl std::cmp::PartialEq for List {
}
}
impl std::cmp::PartialOrd for List {
impl PartialOrd for List {
fn partial_cmp(&self, other: &List) -> Option<Ordering> {
let us = self.cells.lock().unwrap().clone();
let them = other.cells.lock().unwrap().clone();
@ -246,40 +253,3 @@ impl std::cmp::PartialOrd for List {
Some(Ordering::Equal)
}
}
pub struct ListReader {
list: List,
idx: usize,
types: Vec<ColumnType>,
}
impl ListReader {
pub fn new(list: List, name: &str) -> ListReader {
ListReader {
types: vec![ColumnType::new(name, list.element_type())],
list,
idx: 0usize,
}
}
}
impl CrushStream for ListReader {
fn read(&mut self) -> CrushResult<Row> {
self.idx += 1;
Ok(Row::new(vec![self.list.get(self.idx - 1)?]))
}
fn read_timeout(
&mut self,
_timeout: Duration,
) -> Result<Row, crate::lang::pipe::RecvTimeoutError> {
match self.read() {
Ok(r) => Ok(r),
Err(_) => Err(crate::lang::pipe::RecvTimeoutError::Disconnected),
}
}
fn types(&self) -> &[ColumnType] {
&self.types
}
}

28
src/lang/data/scope.rs
View File

@ -1,5 +1,5 @@
use crate::lang::command::{Command, CrushCommand, OutputType, ArgumentDescription};
use crate::lang::errors::{error, mandate, CrushResult, argument_error_legacy};
use crate::lang::errors::{error, mandate, CrushResult, argument_error_legacy, CrushError};
use crate::lang::execution_context::CommandContext;
use crate::lang::help::Help;
use crate::lang::data::r#struct::Struct;
@ -9,6 +9,7 @@ use ordered_map::OrderedMap;
use std::cmp::max;
use std::sync::{Arc, Mutex, MutexGuard};
use std::fmt::{Display, Formatter};
use crate::lang::errors::CrushErrorType::GenericError;
/**
This is where we store variables, including functions.
@ -154,20 +155,20 @@ impl ScopeLoader {
pub struct ScopeData {
/** This is the parent scope used to perform variable name resolution. If a variable lookup
fails in the current scope, it proceeds to this scope. This is usually the scope in which this
scope was *created*.
fails in the current scope, it proceeds to this scope. This is usually the scope in which this
scope was *created*.
Not that when scopes are used as namespaces, they do not use this scope.
Not that when scopes are used as namespaces, they do not use this scope.
*/
pub parent_scope: Option<Scope>,
/** This is the scope in which the current scope was called. Since a closure can be called
from inside any scope, it need not be the same as the parent scope. This scope is the one used
for break/continue loop control, and it is also the scope that builds up the namespace hierarchy. */
from inside any scope, it need not be the same as the parent scope. This scope is the one used
for break/continue loop control, and it is also the scope that builds up the namespace hierarchy. */
pub calling_scope: Option<Scope>,
/** This is a list of scopes that are imported into the current scope. Anything directly inside
one of these scopes is also considered part of this scope. */
one of these scopes is also considered part of this scope. */
pub uses: Vec<Scope>,
/** The actual data of this scope. */
@ -177,11 +178,11 @@ pub struct ScopeData {
pub is_loop: bool,
/** True if this scope should stop execution, i.e. if the continue or break commands have been
called. */
called. */
pub is_stopped: bool,
/** True if this scope can not be further modified. Note that mutable variables in it, e.g.
lists can still be modified. */
lists can still be modified. */
pub is_readonly: bool,
pub name: Option<String>,
@ -351,6 +352,15 @@ impl Scope {
}
}
pub fn get_calling_scope(&self) -> CrushResult<Scope> {
let data = self.lock()?;
if let Some(scope) = &data.calling_scope {
Ok(scope.clone())
} else {
error("Scope not found")
}
}
pub fn do_break(&self) -> CrushResult<bool> {
let mut data = self.lock()?;
if data.is_readonly {

34
src/lang/data/table.rs
View File

@ -58,7 +58,7 @@ impl CrushStream for TableReader {
Ok(self
.rows
.rows
.replace(self.idx - 1, Row::new(vec![Value::Integer(0)])))
.replace(self.idx - 1, Row::new(vec![Value::Empty()])))
}
fn read_timeout(
@ -147,19 +147,18 @@ impl ColumnType {
impl Display for ColumnType {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
self.name.fmt(f)?;
f.write_str("=(")?;
f.write_str("=($")?;
self.cell_type.fmt(f)?;
f.write_str(")")
}
}
pub trait ColumnVec {
fn find_str(&self, needle: &str) -> CrushResult<usize>;
fn find(&self, needle: &[String]) -> CrushResult<usize>;
fn find(&self, needle: &str) -> CrushResult<usize>;
}
impl ColumnVec for &[ColumnType] {
fn find_str(&self, needle: &str) -> CrushResult<usize> {
fn find(&self, needle: &str) -> CrushResult<usize> {
for (idx, field) in self.iter().enumerate() {
if field.name == needle {
return Ok(idx);
@ -177,29 +176,4 @@ impl ColumnVec for &[ColumnType] {
.as_str(),
)
}
fn find(&self, needle_vec: &[String]) -> CrushResult<usize> {
if needle_vec.len() != 1 {
argument_error_legacy("Expected direct field")
} else {
let needle = &needle_vec[0];
for (idx, field) in self.iter().enumerate() {
if &field.name == needle {
return Ok(idx);
}
}
error(
format!(
"Unknown column {}, available columns are {}",
needle,
self.iter()
.map(|t| t.name.to_string())
.collect::<Vec<String>>()
.join(", "),
)
.as_str(),
)
}
}
}

2
src/lang/errors.rs
View File

@ -1,6 +1,6 @@
use std::error::Error;
use std::fmt::Display;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
use CrushErrorType::*;
use std::cmp::{min, max};

4
src/lang/execute.rs
View File

@ -40,7 +40,7 @@ pub fn pup(
},
)?;
cmd.invoke(CommandContext::new(&env, global_state));
cmd.eval(CommandContext::new(&env, global_state).with_output(snd));
global_state.threads().join(global_state.printer());
Ok(())
@ -58,7 +58,7 @@ pub fn string(
) -> CrushResult<()> {
let jobs = global_state.parser().parse(command, &global_env)?;
for job_definition in jobs {
let handle = job_definition.invoke(JobContext::new(
let handle = job_definition.eval(JobContext::new(
empty_channel(),
output.clone(),
global_env.clone(),

29
src/lang/execution_context.rs
View File

@ -11,7 +11,7 @@ use crate::lang::pipe::{
black_hole, empty_channel, InputStream, OutputStream, ValueReceiver, ValueSender,
};
use crate::lang::printer::Printer;
use crate::lang::value::{Value, ValueType};
use crate::lang::value::{Symbol, Value, ValueType};
use crate::util::glob::Glob;
use crate::util::replace::Replace;
use chrono::{DateTime, Duration, Local};
@ -26,7 +26,7 @@ pub trait ArgumentVector {
fn string(&mut self, idx: usize) -> CrushResult<String>;
fn integer(&mut self, idx: usize) -> CrushResult<i128>;
fn float(&mut self, idx: usize) -> CrushResult<f64>;
fn field(&mut self, idx: usize) -> CrushResult<Vec<String>>;
fn symbol(&mut self, idx: usize) -> CrushResult<Symbol>;
fn file(&mut self, idx: usize) -> CrushResult<PathBuf>;
fn command(&mut self, idx: usize) -> CrushResult<Command>;
fn r#type(&mut self, idx: usize) -> CrushResult<ValueType>;
@ -39,7 +39,7 @@ pub trait ArgumentVector {
fn optional_integer(&mut self, idx: usize) -> CrushResult<Option<i128>>;
fn optional_string(&mut self, idx: usize) -> CrushResult<Option<String>>;
fn optional_command(&mut self, idx: usize) -> CrushResult<Option<Command>>;
fn optional_field(&mut self, idx: usize) -> CrushResult<Option<Vec<String>>>;
fn optional_symbol(&mut self, idx: usize) -> CrushResult<Option<Symbol>>;
fn optional_value(&mut self, idx: usize) -> CrushResult<Option<Value>>;
}
@ -132,7 +132,7 @@ impl ArgumentVector for Vec<Argument> {
argument_getter!(string, String, String, "string");
argument_getter!(integer, i128, Integer, "integer");
argument_getter!(float, f64, Float, "float");
argument_getter!(field, Vec<String>, Field, "field");
argument_getter!(symbol, Symbol, Symbol, "field");
argument_getter!(command, Command, Command, "command");
argument_getter!(r#type, ValueType, Type, "type");
argument_getter!(glob, Glob, Glob, "glob");
@ -162,7 +162,7 @@ impl ArgumentVector for Vec<Argument> {
optional_argument_getter!(optional_bool, bool, bool);
optional_argument_getter!(optional_integer, i128, integer);
optional_argument_getter!(optional_string, String, string);
optional_argument_getter!(optional_field, Vec<String>, field);
optional_argument_getter!(optional_symbol, Symbol, symbol);
optional_argument_getter!(optional_command, Command, command);
optional_argument_getter!(optional_value, Value, value);
}
@ -332,7 +332,7 @@ impl CommandContext {
arguments,
this,
global_state: self.global_state,
handle: self.handle.clone(),
handle: self.handle,
}
}
@ -347,7 +347,22 @@ impl CommandContext {
arguments: self.arguments,
this: self.this,
global_state: self.global_state,
handle: self.handle.clone(),
handle: self.handle,
}
}
/**
Return a new Command context that is identical to this one but with a different output sender.
*/
pub fn with_scope(self, scope : Scope) -> CommandContext {
CommandContext {
input: self.input,
output: self.output,
scope,
arguments: self.arguments,
this: self.this,
global_state: self.global_state,
handle: self.handle,
}
}

10
src/lang/files.rs
View File

@ -10,6 +10,10 @@ use std::io::Write;
use std::path::PathBuf;
use std::convert::{TryFrom};
/**
A type representing a set of files. It is used in the signature of builtin commands that
accept files, including globs, regexes, etc.
*/
#[derive(Debug, Clone)]
pub struct Files {
had_entries: bool,
@ -51,6 +55,7 @@ impl Files {
match input.recv()? {
Value::BinaryInputStream(b) => Ok(b),
Value::Binary(b) => Ok(<dyn BinaryReader>::vec(&b)),
Value::String(s) => Ok(<dyn BinaryReader>::vec(s.as_bytes())),
_ => argument_error_legacy("Expected either a file to read or binary pipe io"),
}
} else {
@ -69,7 +74,7 @@ impl Files {
self.files[0].clone(),
))?))
} else {
argument_error_legacy("Expected exactly one desitnation file")
argument_error_legacy("Expected at most one destination file")
}
}
@ -78,7 +83,8 @@ impl Files {
Value::File(p) => self.files.push(p),
Value::Glob(pattern) => pattern.glob_files(&PathBuf::from("."), &mut self.files)?,
Value::Regex(_, re) => re.match_files(&cwd()?, &mut self.files, printer),
value => match value.stream() {
Value::Symbol(f) => self.files.push(PathBuf::from(f)),
value => match value.stream()? {
None => return argument_error_legacy("Expected a file name"),
Some(mut s) => {
let t = s.types();

13
src/lang/global_state.rs
View File

@ -6,9 +6,10 @@ use crate::lang::threads::ThreadStore;
use num_format::{Grouping, SystemLocale};
use std::sync::{Arc, Mutex};
struct StateData {
locale: SystemLocale,
}
/**
A type representing the shared crush state, such as the printer, the running jobs, the running
threads, etc.
*/
#[derive(Clone)]
pub struct GlobalState {
@ -21,6 +22,10 @@ pub struct GlobalState {
parser: Parser,
}
struct StateData {
locale: SystemLocale,
}
#[derive(Clone, Copy)]
pub struct JobId(usize);
@ -47,7 +52,7 @@ pub struct LiveJob {
}
/**
A resource counter. Once it reaches zero, the job is done, and it is removed from the global job
A resource tracker. Once it reaches zero, the job is done, and it is removed from the global job
table.
*/
#[derive(Clone)]

4
src/lang/interactive/mod.rs
View File

@ -16,7 +16,7 @@ use crate::lang::global_state::GlobalState;
use crate::lang::command::Command;
use crate::lang::command_invocation::CommandInvocation;
use crate::lang::value::{ValueDefinition, Value};
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
use crate::lang::execution_context::JobContext;
const DEFAULT_PROMPT: &'static str = "crush# ";
@ -46,7 +46,7 @@ pub fn execute_prompt(
ValueDefinition::Value(Value::Command(prompt), Location::new(0, 0)),
vec![]);
let (snd, recv) = pipe();
cmd.invoke(JobContext::new(
cmd.eval(JobContext::new(
empty_channel(),
snd,
env.clone(),

2
src/lang/interactive/rustyline_helper.rs
View File

@ -57,7 +57,7 @@ impl RustylineHelper {
QuotedString => highlight.get(&Value::string("string_literal")),
Regex => highlight.get(&Value::string("string_literal")),
QuotedFile => highlight.get(&Value::string("file_literal")),
LabelOrWildcard => highlight.get(&Value::string("label")),
StringOrWildcard => highlight.get(&Value::string("label")),
Integer => highlight.get(&Value::string("numeric_literal")),
Float => highlight.get(&Value::string("numeric_literal")),
Field => highlight.get(&Value::string("field")),

15
src/lang/job.rs
View File

@ -4,7 +4,7 @@ use crate::lang::execution_context::{CompileContext, JobContext};
use crate::lang::pipe::pipe;
use std::thread::ThreadId;
use std::fmt::{Display, Formatter};
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
#[derive(Clone)]
pub struct Job {
@ -23,7 +23,7 @@ impl Job {
pub fn can_block(&self, context: &mut CompileContext) -> bool {
if self.commands.len() == 1 {
self.commands[0].can_block(self.commands[0].arguments(), context)
self.commands[0].can_block(context)
} else {
true
}
@ -33,13 +33,13 @@ impl Job {
&self.commands
}
pub fn invoke(&self, context: JobContext) -> CrushResult<Option<ThreadId>> {
pub fn eval(&self, context: JobContext) -> CrushResult<Option<ThreadId>> {
let context = context.running(self.to_string());
let mut input = context.input.clone();
let last_job_idx = self.commands.len() - 1;
for call_def in &self.commands[..last_job_idx] {
let (output, next_input) = pipe();
call_def.invoke(context.with_io(input, output))?;
call_def.eval(context.with_io(input, output))?;
input = next_input;
if context.scope.is_stopped() {
@ -52,15 +52,16 @@ impl Job {
}
let last_call_def = &self.commands[last_job_idx];
last_call_def.invoke(context.with_io(input, context.output.clone())).map_err(|e| e.with_location(self.location))
last_call_def.eval(context.with_io(input, context.output.clone())).map_err(|e| e.with_location(self.location))
}
pub fn as_string(&self) -> Option<String> {
/** Extracts the help message from a closure definition */
pub fn extract_help_message(&self) -> Option<String> {
if self.commands.len() != 1 {
return None;
}
self.commands[0].as_string()
self.commands[0].extract_help_message()
}
}

36
src/lang/lalrparser.lalrpop
View File

@ -1,4 +1,6 @@
use crate::lang::ast::*;
use crate::lang::ast::location::Location;
use crate::lang::ast::tracked_string::TrackedString;
grammar;
@ -102,15 +104,15 @@ ParameterList: Vec<ParameterNode> = {
}
Parameter: ParameterNode = {
<start: @L> <l: LabelOrWildcard> <end: @R> <d: Default> =>
ParameterNode::Parameter(TrackedString::from(l, Location::new(start, end)), None, d),
<start: @L> <l: Identifier> <end: @R> <d: Default> =>
ParameterNode::Parameter(TrackedString::from(&l[1..], Location::new(start, end)), None, d),
<start: @L> <l: LabelOrWildcard> <end: @R> Colon <t: Item> <d: Default> =>
ParameterNode::Parameter(TrackedString::from(l, Location::new(start, end)), Some(t), d),
<start: @L> <l: Identifier> <end: @R> Colon <t: Item> <d: Default> =>
ParameterNode::Parameter(TrackedString::from(&l[1..], Location::new(start, end)), Some(t), d),
"@" <start: @L> <l: LabelOrWildcard> <end: @R> => ParameterNode::Unnamed(TrackedString::from(l, Location::new(start, end))),
"@" <start: @L> <l: Identifier> <end: @R> => ParameterNode::Unnamed(TrackedString::from(&l[1..], Location::new(start, end))),
"@@" <start: @L> <l: LabelOrWildcard> <end: @R> => ParameterNode::Named(TrackedString::from(l, Location::new(start, end))),
"@@" <start: @L> <l: Identifier> <end: @R> => ParameterNode::Named(TrackedString::from(&l[1..], Location::new(start, end))),
}
Default: Option<Node> = {
@ -119,9 +121,11 @@ Default: Option<Node> = {
}
Item: Box<Node> = {
<start: @L> <l: LabelOrWildcard> <end: @R> =>
Node::parse_label_or_wildcard(&TrackedString::from(l, Location::new(start, end))),
<start: @L> <l: StringOrWildcard> <end: @R> =>
Node::parse_string_or_wildcard(&TrackedString::from(l, Location::new(start, end))),
<start: @L> <l: Identifier> <end: @R> =>
Node::parse_identifier(&TrackedString::from(l, Location::new(start, end))),
<start: @L> <l: Regex> <end: @R> =>
Box::from(Node::Regex(TrackedString::from(&l[3..l.len()-1], Location::new(start, end)))),
@ -145,12 +149,12 @@ Item: Box<Node> = {
<start: @L> <l:Flag> <end: @R> =>
Box::from(
Node::Assignment(Box::from(Node::Label(TrackedString::from(&l[2..], Location::new(start+2, end)))),
Node::Assignment(Box::from(Node::Field(TrackedString::from(&l[2..], Location::new(start+2, end)))),
"=".to_string(),
Box::from(Node::Label(TrackedString::from("true", Location::new(start, start+2)))))),
Box::from(Node::Identifier(TrackedString::from("true", Location::new(start, start+2)))))),
<i: Item> "[" <e: Assignment> "]" => Box::from(Node::GetItem(i, e)),
<i: Item> Colon <start: @L> <l: LabelOrWildcard> <end: @R> => Box::from(Node::GetAttr(i, TrackedString::from(&l, Location::new(start, end)))),
<i: Item> Colon <start: @L> <l: StringOrWildcard> <end: @R> => Box::from(Node::GetAttr(i, TrackedString::from(&l, Location::new(start, end)))),
"{" Separator? <s: Signature> <l: JobListWithoutSeparator> "}" => Box::from(Node::Closure(s, l)),
"(" <j:Job> ")" => Box::from(Node::Substitution(j)),
}
@ -169,7 +173,8 @@ Token: TokenNode = {
@L FactorOperator @R => TokenNode::new(TokenType::FactorOperator, <>),
@L TermOperator @R => TokenNode::new(TokenType::TermOperator, <>),
@L QuotedString @R => TokenNode::new(TokenType::QuotedString, <>),
@L LabelOrWildcard @R => TokenNode::new(TokenType::LabelOrWildcard, <>),
@L Identifier @R => TokenNode::new(TokenType::Identifier, <>),
@L StringOrWildcard @R => TokenNode::new(TokenType::StringOrWildcard, <>),
@L QuotedFile @R => TokenNode::new(TokenType::QuotedFile, <>),
@L FileOrWildcard @R => TokenNode::new(TokenType::FileOrWildcard, <>),
@L Flag @R => TokenNode::new(TokenType::Flag, <>),
@ -202,9 +207,10 @@ match {
r"(\*|//)" => FactorOperator,
r"(\+|-)" => TermOperator,
r#""([^\\"]|\\.)*""# => QuotedString,
r"[_a-zA-Z%\?][\._0-9a-zA-Z%\?/]*" => LabelOrWildcard,
r"(\.[\./_0-9a-zA-Z%\?]*|/([\._0-9a-zA-Z%\?][\./_0-9a-zA-Z%\?]*)?)" => FileOrWildcard,
r"--[_0-9a-zA-Z]+" => Flag,
r"[_a-zA-Z%\?][\._0-9a-zA-Z%\?/]*" => StringOrWildcard,
r"\$[_0-9a-zA-Z][_0-9a-zA-Z]*" => Identifier,
r"([\.~][\./_0-9a-zA-Z%\?]*|/([\._0-9a-zA-Z%\?][\./_0-9a-zA-Z%\?]*)?)" => FileOrWildcard,
r"--[_0-9a-zA-Z]*" => Flag,
r"\^[\._a-zA-Z][\._a-zA-Z0-9]*" => Field,
r#"'([^\\']|\\.)*'"# => QuotedFile,
r#"re"([^"]|\\.)*""# => Regex,

3
src/lang/mod.rs
View File

@ -1,3 +1,6 @@
/**
Core language implementation lives in this crate
*/
pub mod argument;
pub mod ast;
pub mod command;

4
src/lang/number.rs
View File

@ -1,3 +1,7 @@
/**
A type representing a f64 or an i128. It is used in the signature of builtin commands that
accept any type of nomeric value as arguments, e.g. the math library.
*/
pub enum Number {
Float(f64),
Integer(i128),

5
src/lang/patterns.rs
View File

@ -3,6 +3,11 @@ use crate::util::glob::Glob;
use regex::Regex;
use std::fmt::{Display, Formatter};
/**
A type representing a set of text patterns. The test method can be used to check if the patterns
match.
*/
pub struct Patterns {
patterns: Vec<Value>,
}

11
src/lang/pipe.rs
View File

@ -174,6 +174,17 @@ pub fn pipe() -> (ValueSender, ValueReceiver) {
)
}
/**
A Sender/Receiver pair that is bounded to only one Value on the wire before blocking.
*/
pub fn printer_pipe() -> (ValueSender, ValueReceiver) {
let (send, recv) = bounded(1);
(
ValueSender { sender: send, is_pipeline: false },
ValueReceiver { receiver: recv, is_pipeline: false },
)
}
pub fn streams(signature: Vec<ColumnType>) -> (OutputStream, InputStream) {
let (output, input) = bounded(128);
(

51
src/lang/pretty.rs
View File

@ -4,9 +4,8 @@
use crate::lang::data::binary::BinaryReader;
use crate::lang::errors::to_crush_error;
use crate::lang::data::list::ListReader;
use crate::lang::printer::Printer;
use crate::lang::pipe::{CrushStream, InputStream, ValueSender, pipe};
use crate::lang::pipe::{CrushStream, InputStream, ValueSender, printer_pipe, Stream};
use crate::lang::data::table::ColumnType;
use crate::lang::data::table::Row;
use crate::lang::data::table::Table;
@ -17,12 +16,34 @@ use crate::lang::value::ValueType;
use crate::lang::data::r#struct::Struct;
use std::cmp::max;
use std::io::{BufReader, Read};
use std::ops::Deref;
use std::thread;
use chrono::Duration;
use crate::util::hex::to_hex;
use crate::lang::global_state::GlobalState;
use num_format::Grouping;
pub fn create_pretty_printer(
printer: Printer,
global_state: &GlobalState,
) -> ValueSender {
let global_state = global_state.clone();
let (o, i) = printer_pipe();
let printer_clone = printer.clone();
printer_clone.handle_error(to_crush_error(
thread::Builder::new()
.name("output-formater".to_string())
.spawn(move || {
let mut pp = PrettyPrinter { printer, grouping: global_state.grouping() };
while let Ok(val) = i.recv() {
pp.grouping = global_state.grouping();
pp.print_value(val);
}
}),
));
o
}
trait Width {
fn width(&self) -> usize;
}
@ -39,26 +60,6 @@ impl Width for &str {
}
}
pub fn create_pretty_printer(
printer: Printer,
global_state: &GlobalState,
) -> ValueSender {
let global_state = global_state.clone();
let (o, i) = pipe();
let printer_clone = printer.clone();
printer_clone.handle_error(to_crush_error(
thread::Builder::new()
.name("output-formater".to_string())
.spawn(move || {
let mut pp = PrettyPrinter { printer, grouping: global_state.grouping() };
while let Ok(val) = i.recv() {
pp.grouping = global_state.grouping();
pp.print_value(val);
}
}),
));
o
}
pub struct PrettyPrinter {
printer: Printer,
@ -136,14 +137,14 @@ impl PrettyPrinter {
if list.len() < 8 {
self.printer.line(list.to_string().as_str())
} else {
self.print_stream(&mut ListReader::new(list, "value"), 0)
self.print_stream(list.stream().as_mut(), 0)
}
}
_ => self.printer.line(cell.to_pretty_string(self.grouping).as_str()),
};
}
fn print_stream(&self, stream: &mut impl CrushStream, indent: usize) {
fn print_stream(&self, stream: &mut dyn CrushStream, indent: usize) {
let mut data: Vec<Row> = Vec::new();
let mut has_table = false;
@ -388,7 +389,7 @@ impl PrettyPrinter {
Value::TableInputStream(mut output) => self.print_stream(&mut output, indent),
Value::Table(rows) => self.print_stream(&mut TableReader::new(rows), indent),
Value::BinaryInputStream(mut b) => self.print_binary(b.as_mut(), indent),
Value::List(list) => self.print_stream(&mut ListReader::new(list, "value"), indent),
Value::List(list) => self.print_stream(list.stream().as_mut(), indent),
_ => {
let mut line = " ".repeat(4 * indent);
line.push_str(&ss);

2
src/lang/printer.rs
View File

@ -16,7 +16,7 @@ use crate::lang::printer::PrinterMessage::*;
use std::thread::JoinHandle;
use termion::terminal_size;
use std::cmp::max;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
#[derive(Clone)]
pub struct Printer {

5
src/lang/serialization/tracked_string_serializer.rs
View File

@ -1,9 +1,10 @@
use crate::lang::errors::{error, CrushResult};
use crate::lang::errors::{CrushResult, error};
use crate::lang::serialization::model::{element, Element};
use crate::lang::serialization::model;
use crate::lang::serialization::{DeserializationState, Serializable, SerializationState};
use crate::lang::ast::{TrackedString, Location};
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::ast::location::Location;
impl Serializable<TrackedString> for TrackedString {
fn deserialize(

6
src/lang/serialization/value_serializer.rs
View File

@ -36,7 +36,7 @@ fn serialize_simple(
Value::Bool(b) => element::Element::Bool(*b),
Value::Empty() => element::Element::Empty(false),
Value::Time(d) => element::Element::Time(d.timestamp_nanos()),
Value::Field(f) => element::Element::Field(f.serialize(elements, state)? as u64),
Value::Symbol(f) => element::Element::Symbol(f.to_string()),
_ => return error("Expected simple value"),
}),
};
@ -88,7 +88,7 @@ impl Serializable<Value> for Value {
id, elements, state,
)?)),
element::Element::Field(f) => Ok(Value::Field(Vec::deserialize(*f as usize, elements, state)?)),
element::Element::Symbol(s) => Ok(Value::Symbol(s.to_string())),
element::Element::UserScope(_) | element::Element::InternalScope(_) => {
Ok(Value::Scope(Scope::deserialize(id, elements, state)?))
}
@ -121,7 +121,7 @@ impl Serializable<Value> for Value {
| Value::Bool(_)
| Value::Empty()
| Value::Time(_)
| Value::Field(_) => serialize_simple(self, elements, state),
| Value::Symbol(_) => serialize_simple(self, elements, state),
Value::Integer(s) => s.serialize(elements, state),

4
src/lang/serialization/value_type_serializer.rs
View File

@ -23,7 +23,7 @@ impl Serializable<ValueType> for ValueType {
4 => ValueType::Command,
5 => ValueType::Binary,
6 => ValueType::Duration,
7 => ValueType::Field,
7 => ValueType::Symbol,
8 => ValueType::Glob,
9 => ValueType::Regex,
10 => ValueType::Scope,
@ -84,7 +84,7 @@ impl Serializable<ValueType> for ValueType {
ValueType::Integer => SimpleTypeKind::Integer,
ValueType::Time => SimpleTypeKind::Time,
ValueType::Duration => SimpleTypeKind::Duration,
ValueType::Field => SimpleTypeKind::Field,
ValueType::Symbol => SimpleTypeKind::Symbol,
ValueType::Glob => SimpleTypeKind::Glob,
ValueType::Regex => SimpleTypeKind::Regex,
ValueType::Command => SimpleTypeKind::Command,

12
src/lang/threads.rs
View File

@ -8,7 +8,9 @@ use crossbeam::channel::Receiver;
use crossbeam::channel::unbounded;
use std::time::Duration;
use chrono::{DateTime, Local};
/**
A thread management utility. Spawn, track and join on threads.
*/
struct ThreadData {
handle: JoinHandle<CrushResult<()>>,
creation_time: DateTime<Local>,
@ -73,10 +75,10 @@ impl ThreadStore {
}))?;
let id = handle.thread().id();
let mut data = self.data.lock().unwrap();
data.threads.push(ThreadData {
handle,
creation_time: Local::now(),
});
data.threads.push(ThreadData {
handle,
creation_time: Local::now(),
});
Ok(id)
}

109
src/lang/value/mod.rs
View File

@ -1,6 +1,6 @@
/**
The type representing all values in crush.
*/
The type representing all values in crush.
*/
mod value_definition;
mod value_type;
@ -12,13 +12,13 @@ use std::str::FromStr;
use chrono::{DateTime, Local};
use regex::Regex;
use crate::lang::errors::{argument_error_legacy, mandate, CrushResult};
use crate::lang::errors::{argument_error_legacy, mandate, CrushResult, data_error, eof_error};
use crate::lang::data::r#struct::Struct;
use crate::lang::data::r#struct::StructReader;
use crate::lang::data::scope::Scope;
use crate::lang::pipe::{streams, InputStream, Stream, OutputStream};
use crate::lang::pipe::{streams, InputStream, Stream, OutputStream, CrushStream};
use crate::lang::data::{
binary::BinaryReader, dict::Dict, dict::DictReader, list::List, list::ListReader,
binary::BinaryReader, dict::Dict, dict::DictReader, list::List,
table::ColumnType, table::TableReader,
};
use crate::util::time::duration_format;
@ -38,18 +38,20 @@ use crate::util::regex::RegexFileMatcher;
use ordered_map::OrderedMap;
pub use value_definition::ValueDefinition;
pub use value_type::ValueType;
use std::fmt::{Display, Formatter, Debug};
use std::fmt::{Display, Formatter};
use num_format::Grouping;
use crate::data::table::Row;
use crate::util::escape::escape;
use crate::util::replace::Replace;
pub type Field = Vec<String>;
pub type Symbol = String;
pub enum Value {
String(String),
Integer(i128),
Time(DateTime<Local>),
Duration(Duration),
Field(Field),
Symbol(Symbol),
Glob(Glob),
Regex(String, Regex),
Command(Command),
@ -75,10 +77,7 @@ impl Display for Value {
Value::String(val) => std::fmt::Display::fmt(val, f),
Value::Integer(val) => std::fmt::Display::fmt(val, f),
Value::Time(val) => f.write_str(&val.format("%Y-%m-%d %H:%M:%S %z").to_string()),
Value::Field(val) => {
f.write_str("^")?;
f.write_str(&val.join(":"))
}
Value::Symbol(val) => std::fmt::Display::fmt(val, f),
Value::Glob(val) => std::fmt::Display::fmt(val, f),
Value::Regex(val, _) => {
f.write_str("re\"")?;
@ -144,6 +143,49 @@ impl From<bool> for Value {
}
}
pub struct VecReader {
vec: Vec<Value>,
types: Vec<ColumnType>,
idx: usize,
}
impl VecReader {
pub fn new(
vec: Vec<Value>,
column_type: ValueType,
) -> VecReader {
VecReader {
vec,
types: vec![ColumnType::new("value", column_type)],
idx: 0,
}
}
}
impl CrushStream for VecReader {
fn read(&mut self) -> CrushResult<Row> {
self.idx += 1;
if self.idx > self.vec.len() {
return eof_error()
}
Ok(Row::new(vec![self.vec.replace(self.idx - 1, Value::Empty())]))
}
fn read_timeout(
&mut self,
_timeout: Duration,
) -> Result<Row, crate::lang::pipe::RecvTimeoutError> {
match self.read() {
Ok(r) => Ok(r),
Err(_) => Err(crate::lang::pipe::RecvTimeoutError::Disconnected),
}
}
fn types(&self) -> &[ColumnType] {
&self.types
}
}
impl Value {
pub fn bind(self, this: Value) -> Value {
match self {
@ -218,15 +260,22 @@ impl Value {
Value::String(s.into())
}
pub fn stream(&self) -> Option<Stream> {
match self {
pub fn stream(&self) -> CrushResult<Option<Stream>> {
Ok(match self {
Value::TableInputStream(s) => Some(Box::from(s.clone())),
Value::Table(r) => Some(Box::from(TableReader::new(r.clone()))),
Value::List(l) => Some(Box::from(ListReader::new(l.clone(), "value"))),
Value::List(l) => Some(l.stream()),
Value::Dict(d) => Some(Box::from(DictReader::new(d.clone()))),
Value::Struct(s) => Some(Box::from(StructReader::new(s.clone()))),
Value::Glob(l) => {
let mut paths = Vec::<PathBuf>::new();
l.glob_files(&cwd()?, &mut paths)?;
Some(Box::from(VecReader::new(
paths.iter().map(|e| { Value::File(e.to_path_buf()) }).collect(),
ValueType::File)))
}
_ => None,
}
})
}
pub fn value_type(&self) -> ValueType {
@ -234,7 +283,7 @@ impl Value {
Value::String(_) => ValueType::String,
Value::Integer(_) => ValueType::Integer,
Value::Time(_) => ValueType::Time,
Value::Field(_) => ValueType::Field,
Value::Symbol(_) => ValueType::Symbol,
Value::Glob(_) => ValueType::Glob,
Value::Regex(_, _) => ValueType::Regex,
Value::Command(_) => ValueType::Command,
@ -262,7 +311,7 @@ impl Value {
Value::File(p) => v.push(p.clone()),
Value::Glob(pattern) => pattern.glob_files(&PathBuf::from("."), v)?,
Value::Regex(_, re) => re.match_files(&cwd()?, v, printer),
val => match val.stream() {
val => match val.stream()? {
None => return error("Expected a file name"),
Some(mut s) => {
let t = s.types();
@ -329,7 +378,7 @@ impl Value {
ValueType::File => Ok(Value::File(PathBuf::from(str_val.as_str()))),
ValueType::Glob => Ok(Value::Glob(Glob::new(str_val.as_str()))),
ValueType::Integer => to_crush_error(str_val.parse::<i128>()).map(Value::Integer),
ValueType::Field => Ok(Value::Field(vec![str_val])),
ValueType::Symbol => Ok(Value::Symbol(str_val)),
ValueType::Regex => {
to_crush_error(Regex::new(str_val.as_str()).map(|v| Value::Regex(str_val, v)))
}
@ -363,13 +412,13 @@ impl Value {
}
/**
Format this value in a way appropriate for use in the pretty printer.
Format this value in a way appropriate for use in the pretty printer.
* Escape non-printable strings
* Respect integer grouping, but use _ intead of whatever number group
separator the locale prescribes, so that the number can be copied
and pasted into the terminal again.
*/
* Escape non-printable strings
* Respect integer grouping, but use _ intead of whatever number group
separator the locale prescribes, so that the number can be copied
and pasted into the terminal again.
*/
pub fn to_pretty_string(&self, grouping: Grouping) -> String {
match self {
Value::String(val) =>
@ -442,7 +491,7 @@ impl Clone for Value {
Value::String(v) => Value::String(v.clone()),
Value::Integer(v) => Value::Integer(*v),
Value::Time(v) => Value::Time(*v),
Value::Field(v) => Value::Field(v.clone()),
Value::Symbol(v) => Value::Symbol(v.clone()),
Value::Glob(v) => Value::Glob(v.clone()),
Value::Regex(v, r) => Value::Regex(v.clone(), r.clone()),
Value::Command(v) => Value::Command(v.as_ref().copy()),
@ -488,7 +537,7 @@ impl std::hash::Hash for Value {
Value::String(v) => v.hash(state),
Value::Integer(v) => v.hash(state),
Value::Time(v) => v.hash(state),
Value::Field(v) => v.hash(state),
Value::Symbol(v) => v.hash(state),
Value::Glob(v) => v.hash(state),
Value::Regex(v, _) => v.hash(state),
Value::Command(_) => {}
@ -530,7 +579,7 @@ impl std::cmp::PartialEq for Value {
(Value::Integer(val1), Value::Integer(val2)) => val1 == val2,
(Value::Time(val1), Value::Time(val2)) => val1 == val2,
(Value::Duration(val1), Value::Duration(val2)) => val1 == val2,
(Value::Field(val1), Value::Field(val2)) => val1 == val2,
(Value::Symbol(val1), Value::Symbol(val2)) => val1 == val2,
(Value::Glob(val1), Value::Glob(val2)) => val1 == val2,
(Value::Regex(val1, _), Value::Regex(val2, _)) => val1 == val2,
(Value::File(val1), Value::String(val2)) => {
@ -569,7 +618,7 @@ impl std::cmp::PartialOrd for Value {
(Value::Integer(val1), Value::Integer(val2)) => Some(val1.cmp(val2)),
(Value::Time(val1), Value::Time(val2)) => Some(val1.cmp(val2)),
(Value::Duration(val1), Value::Duration(val2)) => Some(val1.cmp(val2)),
(Value::Field(val1), Value::Field(val2)) => Some(val1.cmp(val2)),
(Value::Symbol(val1), Value::Symbol(val2)) => Some(val1.cmp(val2)),
(Value::Glob(val1), Value::Glob(val2)) => Some(val1.cmp(val2)),
(Value::Regex(val1, _), Value::Regex(val2, _)) => Some(val1.cmp(val2)),
(Value::File(val1), Value::File(val2)) => Some(val1.cmp(val2)),
@ -631,7 +680,7 @@ mod tests {
assert_eq!(Value::string("1d").convert(ValueType::File).is_err(), false);
assert_eq!(Value::string("1d").convert(ValueType::Time).is_err(), true);
assert_eq!(
Value::string("fad").convert(ValueType::Field).is_err(),
Value::string("fad").convert(ValueType::Symbol).is_err(),
false
);
}

61
src/lang/value/value_definition.rs
View File

@ -1,47 +1,39 @@
use crate::lang::command::Parameter;
use crate::lang::errors::{block_error, mandate};
use crate::lang::errors::mandate;
use crate::lang::execution_context::CompileContext;
use crate::lang::{argument::ArgumentDefinition, command::CrushCommand, job::Job};
use crate::lang::{command::CrushCommand, job::Job};
use crate::{
lang::errors::CrushResult, lang::pipe::pipe, lang::pipe::empty_channel,
lang::errors::CrushResult, lang::pipe::empty_channel, lang::pipe::pipe,
lang::value::Value,
};
use std::path::PathBuf;
use std::fmt::{Display, Formatter};
use crate::lang::ast::{Location, TrackedString};
use crate::lang::ast::tracked_string::TrackedString;
use crate::lang::ast::location::Location;
#[derive(Clone)]
pub enum ValueDefinition {
Value(Value, Location),
ClosureDefinition(Option<TrackedString>, Option<Vec<Parameter>>, Vec<Job>, Location),
JobDefinition(Job),
Label(TrackedString),
Identifier(TrackedString),
GetAttr(Box<ValueDefinition>, TrackedString),
Path(Box<ValueDefinition>, TrackedString),
}
fn file_get(f: &str) -> Option<Value> {
let p = PathBuf::from(f);
if p.exists() {
Some(Value::File(p))
} else {
None
}
}
impl ValueDefinition {
pub fn location(&self) -> Location {
match self {
ValueDefinition::Value(_, l) => *l,
ValueDefinition::ClosureDefinition(_, _, _, l) => *l,
ValueDefinition::JobDefinition(j) => j.location(),
ValueDefinition::Label(l) => l.location,
ValueDefinition::Identifier(l) => l.location,
ValueDefinition::GetAttr(p, a) |
ValueDefinition::Path(p, a)=> p.location().union(a.location),
}
}
pub fn can_block(&self, _arg: &[ArgumentDefinition], context: &mut CompileContext) -> bool {
pub fn can_block(&self, context: &mut CompileContext) -> bool {
match self {
ValueDefinition::JobDefinition(j) => j.can_block(context),
ValueDefinition::GetAttr(_inner1, _inner2) => true,
@ -49,33 +41,21 @@ impl ValueDefinition {
}
}
pub fn compile_unbound(
&self,
context: &mut CompileContext,
) -> CrushResult<(Option<Value>, Value)> {
self.compile_internal(context, true)
}
pub fn compile_bound(&self, context: &mut CompileContext) -> CrushResult<Value> {
let (t, v) = self.compile_internal(context, true)?;
pub fn eval_and_bind(&self, context: &mut CompileContext) -> CrushResult<Value> {
let (t, v) = self.eval(context)?;
Ok(t.map(|tt| v.clone().bind(tt)).unwrap_or(v))
}
pub fn compile_internal(
pub fn eval(
&self,
context: &mut CompileContext,
can_block: bool,
) -> CrushResult<(Option<Value>, Value)> {
Ok(match self {
ValueDefinition::Value(v, _) => (None, v.clone()),
ValueDefinition::JobDefinition(def) => {
let first_input = empty_channel();
let (last_output, last_input) = pipe();
if !can_block {
return block_error();
}
def.invoke(context.job_context(first_input, last_output))?;
def.eval(context.job_context(first_input, last_output))?;
(None, last_input.recv()?)
}
ValueDefinition::ClosureDefinition(name, p, c, _) => (
@ -88,23 +68,20 @@ impl ValueDefinition {
vec![],
)),
),
ValueDefinition::Label(s) => (
ValueDefinition::Identifier(s) => (
None,
mandate(
context.env.get(&s.string)?.or_else(|| file_get(&s.string)),
context.env.get(&s.string)?,
&format!("Unknown variable {}", self),
)?,
),
ValueDefinition::GetAttr(parent_def, entry) => {
let (grand_parent, mut parent) = parent_def.compile_internal(context, can_block)?;
let (grand_parent, mut parent) = parent_def.eval(context)?;
parent = if let Value::Command(parent_cmd) = &parent {
if !can_block {
return block_error();
}
let first_input = empty_channel();
let (last_output, last_input) = pipe();
parent_cmd.invoke(
parent_cmd.eval(
context
.job_context(first_input, last_output)
.command_context(vec![], grand_parent),
@ -125,7 +102,7 @@ impl ValueDefinition {
}
ValueDefinition::Path(parent_def, entry) => {
let parent = parent_def.compile_internal(context, can_block)?.1;
let parent = parent_def.eval(context)?.1;
let val = mandate(
parent.path(&entry.string),
&format!("Missing path entry {} in {}", entry, parent_def),
@ -140,9 +117,9 @@ impl Display for ValueDefinition {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match &self {
ValueDefinition::Value(v, _location) => v.fmt(f),
ValueDefinition::Label(v) => v.fmt(f),
ValueDefinition::Identifier(v) => v.fmt(f),
ValueDefinition::ClosureDefinition(_, _, _, _location) => f.write_str("<closure>"),
ValueDefinition::JobDefinition(_) => f.write_str("<job>"),
ValueDefinition::JobDefinition(j) => j.fmt(f),
ValueDefinition::GetAttr(v, l) => {
v.fmt(f)?;
f.write_str(":")?;

14
src/lang/value/value_type.rs
View File

@ -17,7 +17,7 @@ pub enum ValueType {
Integer,
Time,
Duration,
Field,
Symbol,
Glob,
Regex,
Command,
@ -82,7 +82,7 @@ impl ValueType {
| ValueType::Integer
| ValueType::Time
| ValueType::Duration
| ValueType::Field
| ValueType::Symbol
| ValueType::Glob
| ValueType::Regex
| ValueType::Command
@ -131,7 +131,7 @@ impl ValueType {
Ok(n) => Ok(Value::Integer(n)),
Err(e) => error(e.to_string().as_str()),
},
ValueType::Field => Ok(Value::Field(mandate(parse_name(s), "Invalid field name")?)),
ValueType::Symbol => Ok(Value::Symbol(s.to_string())),
ValueType::Glob => Ok(Value::Glob(Glob::new(s))),
ValueType::Regex => Ok(Value::Regex(s.to_string(), to_crush_error(Regex::new(s))?)),
ValueType::File => Ok(Value::string(s)),
@ -155,7 +155,7 @@ impl Help for ValueType {
ValueType::Integer => "A numeric type representing an integer number.",
ValueType::Time => "A point in time with nanosecond precision",
ValueType::Duration => "A difference between two points in time",
ValueType::Field => "A field is used to represent a path into a datastructure",
ValueType::Symbol => "A field is used to represent a path into a datastructure",
ValueType::Glob => "A pattern containing wildcards",
ValueType::Regex => "An advanced pattern that can be used for matching and replacing",
ValueType::Command => "A piece fo code that can be called",
@ -184,8 +184,8 @@ impl Help for ValueType {
let mut lines = match self {
ValueType::Time => {
vec![" All time instances use the local time zone.\n".to_string()]
},
_ => {Vec::new()}
}
_ => { Vec::new() }
};
let mut keys: Vec<_> = self.fields().into_iter().collect();
@ -218,7 +218,7 @@ impl Display for ValueType {
ValueType::Integer => f.write_str("integer"),
ValueType::Time => f.write_str("time"),
ValueType::Duration => f.write_str("duration"),
ValueType::Field => f.write_str("field"),
ValueType::Symbol => f.write_str("field"),
ValueType::Glob => f.write_str("glob"),
ValueType::Regex => f.write_str("regex"),
ValueType::Command => f.write_str("command"),

4
src/lib/cond.rs
View File

@ -17,7 +17,7 @@ pub fn and(mut context: CommandContext) -> CrushResult<()> {
Value::Command(c) => {
let (sender, receiver) = pipe();
let cc = context.empty().with_output(sender);
c.invoke(cc)?;
c.eval(cc)?;
match receiver.recv()? {
Value::Bool(b) => {
if !b {
@ -48,7 +48,7 @@ pub fn or(mut context: CommandContext) -> CrushResult<()> {
Value::Command(c) => {
let (sender, receiver) = pipe();
let cc = context.empty().with_output(sender);
c.invoke(cc)?;
c.eval(cc)?;
match receiver.recv()? {
Value::Bool(b) => {
if b {

21
src/lib/control/for.rs
View File

@ -3,16 +3,30 @@ use crate::lang::errors::{mandate, CrushResult};
use crate::lang::execution_context::{ArgumentVector, CommandContext};
use crate::lang::value::Value;
use crate::lang::data::r#struct::Struct;
use crate::lang::value::ValueType;
use crate::lang::data::table::ColumnType;
use lazy_static::lazy_static;
use crate::data::table::Row;
use crate::lang::command::OutputType::Known;
use crate::lang::pipe::pipe;
lazy_static! {
static ref OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("value", ValueType::Any),
];
}
pub fn r#for(mut context: CommandContext) -> CrushResult<()> {
context.output.send(Value::Empty())?;
let output = context.output.initialize(OUTPUT_TYPE.clone())?;
let (sender, receiver) = pipe();
context.arguments.check_len(2)?;
let location = context.arguments[0].location;
let body = context.arguments.command(1)?;
let iter = context.arguments.remove(0);
let name = iter.argument_type;
let mut input = mandate(iter.value.stream(), "Expected a stream")?;
let mut input = mandate(iter.value.stream()?, "Expected a stream")?;
while let Ok(line) = input.read() {
let env = context.scope.create_child(&context.scope, true);
@ -38,7 +52,8 @@ pub fn r#for(mut context: CommandContext) -> CrushResult<()> {
}
}
};
body.invoke(context.empty())?;
body.eval(context.empty().with_scope(env.clone()).with_args(arguments, None).with_output(sender.clone()))?;
output.send(Row::new(vec![receiver.recv()?]))?;
if env.is_stopped() {
break;
}

4
src/lib/control/if.rs
View File

@ -22,10 +22,10 @@ fn r#if(context: CommandContext) -> CrushResult<()> {
let cfg: If = If::parse(context.arguments.clone(), &context.global_state.printer())?;
if cfg.condition {
cfg.true_clause.invoke(context.with_args(vec![], None))
cfg.true_clause.eval(context.with_args(vec![], None))
} else {
cfg.false_clause
.map(|v| v.invoke(context.with_args(vec![], None)))
.map(|v| v.eval(context.with_args(vec![], None)))
.unwrap_or(Ok(()))
}
}

19
src/lib/control/loop.rs
View File

@ -1,11 +1,24 @@
use crate::lang::command::Command;
use crate::lang::errors::CrushResult;
use crate::lang::execution_context::CommandContext;
use crate::lang::value::ValueType;
use crate::lang::data::table::ColumnType;
use lazy_static::lazy_static;
use crate::data::table::Row;
use crate::lang::pipe::pipe;
use crate::lang::command::OutputType::Known;
use signature::signature;
lazy_static! {
static ref OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("value", ValueType::Any),
];
}
#[signature(
r#loop,
condition = true,
output = Known(ValueType::TableInputStream(OUTPUT_TYPE.clone())),
short = "Repeatedly execute the body until the break command is called.",
example = "loop {\n if (i_am_tired) {\n break\n }\n echo \"Working\"\n }"
)]
@ -16,10 +29,12 @@ pub struct Loop {
fn r#loop(context: CommandContext) -> CrushResult<()> {
let cfg: Loop = Loop::parse(context.arguments.clone(), &context.global_state.printer())?;
context.output.initialize(vec![])?;
let output = context.output.initialize(OUTPUT_TYPE.clone())?;
let (sender, receiver) = pipe();
loop {
let env = context.scope.create_child(&context.scope, true);
cfg.body.invoke(context.empty())?;
cfg.body.eval(context.empty().with_scope(env.clone()).with_output(sender.clone()))?;
output.send(Row::new(vec![receiver.recv()?]))?;
if env.is_stopped() {
break;
}

2
src/lib/control/mod.rs
View File

@ -193,7 +193,7 @@ example = "pipe := ((table_input_stream value=integer):pipe)\n _1 := (seq 100
struct Fg {}
fn fg(context: CommandContext) -> CrushResult<()> {
let mut result_stream = mandate(context.input.recv()?.stream(), "Invalid input")?;
let mut result_stream = mandate(context.input.recv()?.stream()?, "Invalid input")?;
let mut result: Vec<Value> = result_stream.read()?.into();
if result.len() != 1 {
data_error("Expected a single row, single column result")

20
src/lib/control/while.rs
View File

@ -2,12 +2,24 @@ use crate::lang::command::Command;
use crate::lang::errors::{data_error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::value::Value;
use crate::lang::value::ValueType;
use crate::lang::data::table::ColumnType;
use lazy_static::lazy_static;
use crate::data::table::Row;
use signature::signature;
use crate::lang::command::OutputType::Known;
use crate::lang::pipe::pipe;
lazy_static! {
static ref OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("value", ValueType::Any),
];
}
#[signature(
r#while,
condition = true,
output = Known(ValueType::TableInputStream(OUTPUT_TYPE.clone())),
short = "Repeatedly execute the body for as long the condition is met.",
long = "The loop body is optional. If not specified, the condition is executed until it returns false.\n This effectively means that the condition becomes the body, and the loop break check comes at\n the end of the loop.",
example = "while {./some_file:exists} {echo \"hello\"}"
@ -20,14 +32,15 @@ pub struct While {
}
fn r#while(mut context: CommandContext) -> CrushResult<()> {
context.output.initialize(vec![])?;
let output = context.output.initialize(OUTPUT_TYPE.clone())?;
let (body_sender, body_receiver) = pipe();
let cfg: While = While::parse(context.remove_arguments(), &context.global_state.printer())?;
loop {
let (sender, receiver) = pipe();
let cond_env = context.scope.create_child(&context.scope, true);
cfg.condition.invoke(context.empty().with_output(sender))?;
cfg.condition.eval(context.empty().with_scope(cond_env.clone()).with_output(sender))?;
if cond_env.is_stopped() {
break;
}
@ -36,7 +49,8 @@ fn r#while(mut context: CommandContext) -> CrushResult<()> {
Value::Bool(true) => match &cfg.body {
Some(body) => {
let body_env = context.scope.create_child(&context.scope, true);
body.invoke(context.empty())?;
body.eval(context.empty().with_scope(body_env.clone()).with_output(body_sender.clone()))?;
output.send(Row::new(vec![body_receiver.recv()?]))?;
if body_env.is_stopped() {
break;
}

35
src/lib/crush.rs
View File

@ -9,6 +9,7 @@ use nix::unistd::Pid;
use crate::lang::data::dict::Dict;
use std::env;
use lazy_static::lazy_static;
use crate::data::list::List;
use crate::lang::command::Command;
fn make_env() -> Value {
@ -19,6 +20,11 @@ fn make_env() -> Value {
Value::Dict(e)
}
fn make_arguments() -> Value {
Value::List(List::new(ValueType::String, env::args().map(|a| {Value::string(a)}).collect()))
}
lazy_static! {
static ref THREADS_OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("created", ValueType::Time),
@ -65,6 +71,33 @@ fn prompt(context: CommandContext) -> CrushResult<()> {
context.output.send(Value::Empty())
}
lazy_static! {
static ref JOBS_OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("id", ValueType::Integer),
ColumnType::new("description", ValueType::String),
];
}
#[signature(
jobs,
can_block = false,
short = "List running jobs",
output = Known(ValueType::TableInputStream(JOBS_OUTPUT_TYPE.clone())),
long = "All currently running jobs")]
struct Jobs {}
fn jobs(context: CommandContext) -> CrushResult<()> {
let output = context.output.initialize(JOBS_OUTPUT_TYPE.clone())?;
for job in context.global_state.jobs() {
output.send(Row::new(vec![
Value::Integer(usize::from(job.id) as i128),
Value::string(job.description),
]))?;
}
Ok(())
}
mod locale {
use super::*;
use num_format::SystemLocale;
@ -165,9 +198,11 @@ pub fn declare(root: &Scope) -> CrushResult<()> {
crush.declare("highlight", Value::Dict(highlight))?;
crush.declare("env", make_env())?;
crush.declare("arguments", make_arguments())?;
Prompt::declare(crush)?;
Threads::declare(crush)?;
Exit::declare(crush)?;
Jobs::declare(crush)?;
crush.create_namespace(
"locale",

2
src/lib/dbus.rs
View File

@ -17,7 +17,7 @@ use std::convert::TryFrom;
use std::iter::Peekable;
use std::str::Chars;
use std::time::Duration;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
struct DBusThing {
connection: Connection,

6
src/lib/fs/mod.rs
View File

@ -1,5 +1,5 @@
use crate::lang::command::OutputType::Known;
use crate::lang::errors::{to_crush_error, CrushResult};
use crate::lang::errors::{to_crush_error, CrushResult, error};
use crate::lang::execution_context::CommandContext;
use crate::lang::help::Help;
use crate::lang::printer::Printer;
@ -104,6 +104,10 @@ members of a value, write "dir <value>".
}
Some(v) => {
match v {
Value::Symbol(f) => match &context.scope.get_calling_scope()?.get(&f)? {
None => error(format!("Unknown identifier {}", &f))?,
Some(v) => halp(v, &context.global_state.printer()),
},
Value::Command(cmd) => halp(cmd.help(), &context.global_state.printer()),
Value::Type(t) => halp(&t, &context.global_state.printer()),
v => halp(&v, &context.global_state.printer()),

67
src/lib/host.rs
View File

@ -1,4 +1,4 @@
use crate::lang::errors::{to_crush_error, CrushResult};
use crate::lang::errors::{to_crush_error, CrushResult, error};
use crate::lang::execution_context::CommandContext;
use crate::lang::data::r#struct::Struct;
use crate::lang::data::scope::Scope;
@ -12,7 +12,13 @@ use battery::State;
use chrono::Duration;
use crate::lang::command::OutputType::Known;
#[signature(name, can_block = false, short = "name of this host")]
extern crate uptime_lib;
#[signature(
name,
can_block = false,
output = Known(ValueType::String),
short = "name of this host")]
struct Name {}
fn name(context: CommandContext) -> CrushResult<()> {
@ -34,14 +40,18 @@ lazy_static! {
];
}
#[signature(uptime, can_block = false, short = "uptime of this host")]
#[signature(
uptime,
can_block = false,
output = Known(ValueType::Duration),
short = "uptime of this host")]
struct Uptime {}
fn uptime(context: CommandContext) -> CrushResult<()> {
Ok(())
// context
// .output
// .send(Value::Duration(Duration::seconds(to_crush_error(psutil::host::uptime()))))
match uptime_lib::get() {
Ok(d) => context.output.send(Value::Duration(Duration::nanoseconds(i64::try_from(d.as_nanos()).unwrap()))),
Err(e) => error(e),
}
}
@ -86,10 +96,14 @@ fn battery(context: CommandContext) -> CrushResult<()> {
Ok(())
}
#[signature(mem, can_block = false, short = "memory usage of this host.")]
struct Mem {}
#[signature(
memory,
can_block = false,
output = Known(ValueType::Struct),
short = "memory usage of this host.")]
struct Memory {}
fn mem(context: CommandContext) -> CrushResult<()> {
fn memory(context: CommandContext) -> CrushResult<()> {
let mem = to_crush_error(sys_info::mem_info())?;
context.output.send(Value::Struct(Struct::new(
vec![
@ -114,7 +128,11 @@ fn mem(context: CommandContext) -> CrushResult<()> {
mod os {
use super::*;
#[signature(name, can_block = false, short = "name of the operating system")]
#[signature(
name,
can_block = false,
output = Known(ValueType::String),
short = "name of the operating system")]
pub struct Name {}
fn name(context: CommandContext) -> CrushResult<()> {
@ -124,9 +142,10 @@ mod os {
}
#[signature(
version,
can_block = false,
short = "version of the operating system kernel"
version,
can_block = false,
output = Known(ValueType::String),
short = "version of the operating system kernel"
)]
pub struct Version {}
@ -140,7 +159,11 @@ mod os {
mod cpu {
use super::*;
#[signature(count, can_block = false, short = "number of CPU cores")]
#[signature(
count,
can_block = false,
output = Known(ValueType::Integer),
short = "number of CPU cores")]
pub struct Count {}
fn count(context: CommandContext) -> CrushResult<()> {
@ -149,7 +172,11 @@ mod cpu {
.send(Value::Integer(to_crush_error(sys_info::cpu_num())? as i128))
}
#[signature(load, can_block = false, short = "current CPU load")]
#[signature(
load,
can_block = false,
output = Known(ValueType::Struct),
short = "current CPU load")]
pub struct Load {}
fn load(context: CommandContext) -> CrushResult<()> {
@ -164,7 +191,11 @@ mod cpu {
)))
}
#[signature(speed, can_block = false, short = "current CPU frequency")]
#[signature(
speed,
can_block = false,
output = Known(ValueType::Integer),
short = "current CPU frequency")]
pub struct Speed {}
fn speed(context: CommandContext) -> CrushResult<()> {
@ -180,7 +211,7 @@ pub fn declare(root: &Scope) -> CrushResult<()> {
"Metadata about this host",
Box::new(move |host| {
Battery::declare(host)?;
Mem::declare(host)?;
Memory::declare(host)?;
Name::declare(host)?;
Uptime::declare(host)?;
host.create_namespace(

2
src/lib/io/csv.rs
View File

@ -66,7 +66,7 @@ fn from(context: CommandContext) -> CrushResult<()> {
skipped += 1;
continue;
}
let line_without_newline = &line[0..line.len() - 1];
let line_without_newline = if line.ends_with('\n') {&line[0..line.len() - 1]} else {&line};
let mut split: Vec<&str> = line_without_newline
.split(separator)
.map(|s| trim.map(|c| s.trim_matches(c)).unwrap_or(s))

6
src/lib/io/json.rs
View File

@ -1,8 +1,6 @@
use crate::lang::execution_context::CommandContext;
use crate::{
lang::errors::CrushError,
lang::{data::table::Row, value::Value, value::ValueType},
};
use crate::lang::errors::CrushError;
use crate::lang::{data::table::Row, value::Value, value::ValueType};
use std::io::{BufReader, Write};
use crate::lang::command::OutputType::Unknown;

2
src/lib/io/lines.rs
View File

@ -64,7 +64,7 @@ struct To {
pub fn to(context: CommandContext) -> CrushResult<()> {
let cfg: To = To::parse(context.arguments, &context.global_state.printer())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let mut out = cfg.file.writer(context.output)?;
if input.types().len() != 1 || input.types()[0].cell_type != ValueType::String {

10
src/lib/io/mod.rs
View File

@ -3,7 +3,7 @@ use crate::lang::errors::{argument_error_legacy, data_error, mandate, CrushResul
use crate::lang::data::list::List;
use crate::lang::pretty::PrettyPrinter;
use crate::lang::data::scope::Scope;
use crate::lang::value::{Field, ValueType};
use crate::lang::value::ValueType;
use crate::lang::{execution_context::CommandContext, value::Value};
use signature::signature;
use rustyline::Editor;
@ -87,11 +87,11 @@ fn echo(context: CommandContext) -> CrushResult<()> {
member,
can_block = false,
short = "Extracts one member from the input struct.",
example = "http \"example.com\" | member ^body | json:from"
example = "http \"example.com\" | member body | json:from"
)]
struct Member {
#[description("the member to extract.")]
field: Field,
field: String,
}
fn member(context: CommandContext) -> CrushResult<()> {
@ -101,8 +101,8 @@ fn member(context: CommandContext) -> CrushResult<()> {
}
match context.input.recv()? {
Value::Struct(s) => context.output.send(mandate(
s.get(&cfg.field[0]),
format!("Unknown field \"{}\"", cfg.field[0]).as_str(),
s.get(&cfg.field),
format!("Unknown field \"{}\"", cfg.field).as_str(),
)?),
_ => data_error("Expected a struct"),
}

2
src/lib/io/pup.rs
View File

@ -12,7 +12,7 @@ to,
can_block = true,
output = Unknown,
short = "Serialize to pup format",
long = "Pup is the native crush serialization format. All pup types, including",
long = "Pup is the native crush serialization format. All Crush types, including",
long = "lambdas can be serialized to this format.",
example = "ls | pup:to")]
struct To {

5
src/lib/io/split.rs
View File

@ -12,7 +12,7 @@ use std::io::{BufRead, BufReader};
#[signature(
from,
can_block = true,
short = "Read specified files (or input) as a table, split on the specified separator"
short = "Read specified files (or input) as a table, split on the specified separator characters.",
)]
struct From {
#[unnamed()]
@ -22,7 +22,8 @@ struct From {
separator: String,
#[description("characters to trim from start and end of each token.")]
trim: Option<String>,
#[description("if false, discard empty tokens.")]
#[default(false)]
#[description("allow empty tokens.")]
allow_empty: bool,
}

70
src/lib/proc.rs
View File

@ -1,10 +1,8 @@
use lazy_static::lazy_static;
use crate::lang::command::OutputType::Known;
use crate::lang::errors::{to_crush_error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::data::scope::Scope;
use crate::lang::data::table::ColumnType;
use crate::{data::table::Row, lang::value::Value, lang::value::ValueType};
use crate::{lang::value::Value, lang::value::ValueType};
use nix::sys::signal;
use nix::unistd::Pid;
use signature::signature;
@ -23,7 +21,7 @@ mod macos {
use signature::signature;
lazy_static! {
static ref PS_OUTPUT_TYPE: Vec<ColumnType> = vec![
static ref LIST_OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("pid", ValueType::Integer),
ColumnType::new("ppid", ValueType::Integer),
ColumnType::new("user", ValueType::String),
@ -58,12 +56,12 @@ mod macos {
}
#[signature(
ps,
list,
can_block = true,
short = "Return a table stream containing information on all running processes on the system",
output = Known(ValueType::TableInputStream(PS_OUTPUT_TYPE.clone())),
long = "ps accepts no arguments.")]
pub struct Ps {}
output = Known(ValueType::TableInputStream(LIST_OUTPUT_TYPE.clone())),
long = "proc:list accepts no arguments.")]
pub struct List {}
use libproc::libproc::bsd_info::BSDInfo;
use libproc::libproc::file_info::{pidfdinfo, ListFDs, ProcFDType};
@ -124,11 +122,11 @@ mod macos {
TaskAllInfo { pbsd, ptinfo }
}
fn ps(context: CommandContext) -> CrushResult<()> {
fn list(context: CommandContext) -> CrushResult<()> {
let mut base_procs = Vec::new();
let arg_max = 2048;//get_arg_max();
let output = context.output.initialize(PS_OUTPUT_TYPE.clone())?;
let output = context.output.initialize(LIST_OUTPUT_TYPE.clone())?;
let users = create_user_map()?;
let mut info: mach_timebase_info = mach_timebase_info{numer: 0, denom: 0};
@ -214,8 +212,8 @@ mod macos {
Value::Integer(i128::from(curr_task.ptinfo.pti_virtual_size)),
Value::Duration(Duration::nanoseconds(
i64::try_from(curr_task.ptinfo.pti_total_user + curr_task.ptinfo.pti_total_system)? *
i64::try_from(info.numer)? /
i64::try_from(info.denom)?)),
i64::from(info.numer) /
i64::from(info.denom))),
Value::String(name)
]));
}
@ -243,7 +241,7 @@ mod linux {
use std::collections::HashMap;
lazy_static! {
static ref PS_OUTPUT_TYPE: Vec<ColumnType> = vec![
static ref LIST_OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("pid", ValueType::Integer),
ColumnType::new("ppid", ValueType::Integer),
ColumnType::new("status", ValueType::String),
@ -276,16 +274,16 @@ mod linux {
#[signature(
ps,
list,
can_block = true,
short = "Return a table stream containing information on all running processes on the system",
output = Known(ValueType::TableInputStream(PS_OUTPUT_TYPE.clone())),
long = "ps accepts no arguments.")]
pub struct Ps {}
output = Known(ValueType::TableInputStream(LIST_OUTPUT_TYPE.clone())),
long = "proc:list accepts no arguments.")]
pub struct List {}
fn ps(context: CommandContext) -> CrushResult<()> {
Ps::parse(context.arguments.clone(), &context.global_state.printer())?;
let output = context.output.initialize(PS_OUTPUT_TYPE.clone())?;
fn list(context: CommandContext) -> CrushResult<()> {
List::parse(context.arguments.clone(), &context.global_state.printer())?;
let output = context.output.initialize(LIST_OUTPUT_TYPE.clone())?;
let users = create_user_map()?;
match psutil::process::processes() {
@ -349,32 +347,6 @@ fn kill(context: CommandContext) -> CrushResult<()> {
context.output.send(Value::Empty())
}
lazy_static! {
static ref JOBS_OUTPUT_TYPE: Vec<ColumnType> = vec![
ColumnType::new("id", ValueType::Integer),
ColumnType::new("description", ValueType::String),
];
}
#[signature(
jobs,
can_block = false,
short = "List running jobs",
output = Known(ValueType::TableInputStream(JOBS_OUTPUT_TYPE.clone())),
long = "All currently running jobs")]
struct Jobs {}
fn jobs(context: CommandContext) -> CrushResult<()> {
let output = context.output.initialize(JOBS_OUTPUT_TYPE.clone())?;
for job in context.global_state.jobs() {
output.send(Row::new(vec![
Value::Integer(usize::from(job.id) as i128),
Value::string(job.description),
]))?;
}
Ok(())
}
pub fn declare(root: &Scope) -> CrushResult<()> {
let e = root.create_namespace(
@ -382,13 +354,11 @@ pub fn declare(root: &Scope) -> CrushResult<()> {
"Process related commands",
Box::new(move |env| {
#[cfg(target_os = "linux")]
macos::Ps::declare(env)?;
linux::List::declare(env)?;
#[cfg(target_os = "macos")]
macos::Ps::declare(env)?;
macos::List::declare(env)?;
Kill::declare(env)?;
Jobs::declare(env)?;
Ok(())
}))?;
root.r#use(&e);
Ok(())
}

8
src/lib/random.rs
View File

@ -2,12 +2,15 @@ use crate::lang::errors::CrushResult;
use crate::lang::execution_context::CommandContext;
use crate::lang::data::scope::Scope;
use crate::lang::value::Value;
use crate::lang::value::ValueType;
use crate::lang::command::OutputType::Known;
use signature::signature;
#[signature(
float,
can_block = false,
short = "generate a random floating point number between 0 (inclusive) and 1 (exclusive)"
short = "generate a random floating point number between 0 (inclusive) and 1 (exclusive)",
output = Known(ValueType::Float),
)]
struct Float {
#[default(1.0)]
@ -26,7 +29,8 @@ fn float(context: CommandContext) -> CrushResult<()> {
#[signature(
integer,
can_block = false,
short = "generate a random integer between 0 and 1 (or some other specified number)"
short = "generate a random integer between 0 and 1 (or some other specified number)",
output = Known(ValueType::Integer),
)]
struct Integer {
#[default(2)]

2
src/lib/stream/count.rs
View File

@ -17,7 +17,7 @@ pub fn count(context: CommandContext) -> CrushResult<()> {
Value::Table(r) => context.output.send(Value::Integer(r.rows().len() as i128)),
Value::List(r) => context.output.send(Value::Integer(r.len() as i128)),
Value::Dict(r) => context.output.send(Value::Integer(r.len() as i128)),
v => match v.stream() {
v => match v.stream()? {
Some(mut input) => {
let mut res: i128 = 0;
while let Ok(_) = input.read() {

8
src/lib/stream/drop.rs
View File

@ -3,7 +3,7 @@ use crate::lang::execution_context::CommandContext;
use crate::lang::data::table::{Row, ColumnVec};
use signature::signature;
use crate::lang::command::OutputType::Unknown;
use crate::lang::value::Field;
use crate::lang::value::Symbol;
use std::collections::HashSet;
#[signature(
@ -11,17 +11,17 @@ drop,
can_block = true,
short = "Drop all fields mentioned from input, copy remainder of input",
long = "This command is does the opposite of the select command.\n It copies all column except the ones specified from input to output.",
example= "ps | drop ^vms ^rss # Drop memory usage columns from output of ps",
example= "ps | drop vms rss # Drop memory usage columns from output of ps",
output = Unknown,
)]
pub struct Drop {
#[unnamed()]
drop: Vec<Field>,
drop: Vec<Symbol>,
}
fn drop(context: CommandContext) -> CrushResult<()> {
let cfg: Drop = Drop::parse(context.arguments.clone(), &context.global_state.printer())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let t = input.types();
let drop = cfg.drop.iter()

9
src/lib/stream/each.rs
View File

@ -1,13 +1,12 @@
use crate::lang::command::Command;
use crate::lang::errors::{error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::pipe::{black_hole, empty_channel};
use crate::lang::{argument::Argument, data::table::ColumnType};
use crate::lang::{data::table::Row, value::Value};
use signature::signature;
use crate::lang::value::ValueType::Empty;
use crate::lang::command::OutputType::Known;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
#[signature(
r#each,
@ -15,7 +14,7 @@ can_block = true,
output = Known(Empty),
short = "Runs a command one for each row of input",
long = "The columns of the row are exported to the environment using the column names.",
example = "ps | where {status != \"Sleeping\"} | each {echo (\"{} is sleepy\":format name)}")]
example = "ps | where {status != \"Sleeping\"} | each {echo (\"{} is sleepy\":format $name)}")]
pub struct Each {
#[description("the command to run.")]
body: Command,
@ -34,7 +33,7 @@ fn run(
.map(|(c, t)| Argument::named(t.name.as_ref(), c, location))
.collect();
condition.invoke(
condition.eval(
base_context
.clone()
.with_args(arguments, None)
@ -46,7 +45,7 @@ pub fn each(context: CommandContext) -> CrushResult<()> {
let location = context.arguments[0].location;
context.output.send(Value::Empty())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let base_context = context.empty();

13
src/lib/stream/enumerate.rs
View File

@ -6,22 +6,29 @@ use signature::signature;
#[signature(enumerate, short = "Prepend a column containing the row number to each row of the input.")]
pub struct Enumerate {
#[description("the index to use for the first row.")]
#[default(0)]
start_index: i128,
#[description("the step between rows.")]
#[default(1)]
step: i128,
}
fn enumerate(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
let cfg: Enumerate = Enumerate::parse(context.arguments, &context.global_state.printer())?;
match context.input.recv()?.stream()? {
Some(mut input) => {
let mut output_type = vec![
ColumnType::new("idx", ValueType::Integer)];
output_type.extend(input.types().to_vec());
let output = context.output.initialize(output_type)?;
let mut line: i128 = 0;
let mut line: i128 = cfg.start_index;
while let Ok(row) = input.read() {
let mut out = vec![Value::Integer(line)];
out.extend(Vec::from(row));
output.send(Row::new(out))?;
line += 1;
line += cfg.step;
}
Ok(())
}

12
src/lib/stream/group.rs
View File

@ -7,7 +7,7 @@ use crate::lang::data::scope::Scope;
use crate::lang::pipe::{pipe, InputStream};
use crate::lang::data::table::ColumnType;
use crate::lang::data::table::ColumnVec;
use crate::lang::value::Field;
use crate::lang::value::Symbol;
use crate::{
lang::errors::argument_error_legacy,
lang::pipe::{unlimited_streams, OutputStream},
@ -23,12 +23,12 @@ use crate::lang::global_state::GlobalState;
group,
can_block = true,
short = "Group stream by the specified column(s)",
example = "find . | group ^user ^type file_count={count} size={sum ^size}"
example = "find . | group user type file_count={count} size={sum size}"
)]
pub struct Group {
#[unnamed()]
#[description("the column(s) to group by and copy into the output stream.")]
group_by: Vec<Field>,
group_by: Vec<Symbol>,
#[named()]
#[description("create these additional columns by aggregating the grouped rows using the supplied aggregation command.")]
command: OrderedStringMap<Command>,
@ -53,7 +53,7 @@ fn aggregate(
let (output_sender, output_receiver) = pipe();
input_sender.send(Value::TableInputStream(rows))?;
drop(input_sender);
commands[0].invoke(
commands[0].eval(
CommandContext::new(&scope, &global_state)
.with_input(input_receiver)
.with_output(output_sender)
@ -74,7 +74,7 @@ fn aggregate(
let local_scope = scope.clone();
let local_state = global_state.clone();
threads.spawn("group:aggr", move ||
local_command.invoke(
local_command.eval(
CommandContext::new(&local_scope, &local_state)
.with_input(input_receiver)
.with_output(output_sender)))?;
@ -141,7 +141,7 @@ fn create_worker_thread(
pub fn group(context: CommandContext) -> CrushResult<()> {
let cfg: Group = Group::parse(context.arguments, &context.global_state.printer())?;
let mut input = mandate(
context.input.recv()?.stream(),
context.input.recv()?.stream()?,
"Expected input to be a stream",
)?;
let input_type = input.types().to_vec();

2
src/lib/stream/head.rs
View File

@ -17,7 +17,7 @@ pub struct Head {
fn head(context: CommandContext) -> CrushResult<()> {
let cfg: Head = Head::parse(context.arguments, &context.global_state.printer())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let output = context.output.initialize(input.types().to_vec())?;
let mut count = 0;

8
src/lib/stream/join.rs
View File

@ -72,11 +72,11 @@ pub fn join(mut context: CommandContext) -> CrushResult<()> {
context.arguments.check_len(2)?;
let l = context.arguments.remove(0);
let r = context.arguments.remove(0);
match (l.argument_type, mandate(l.value.stream(), "Expected a stream")?,
r.argument_type, mandate(r.value.stream(), "Expected a stream")?) {
match (l.argument_type, mandate(l.value.stream()?, "Expected a stream")?,
r.argument_type, mandate(r.value.stream()?, "Expected a stream")?) {
(Some(left_name), left_stream, Some(right_name), right_stream) => {
let left_idx = left_stream.types().find_str(&left_name)?;
let right_idx = right_stream.types().find_str(&right_name)?;
let left_idx = left_stream.types().find(&left_name)?;
let right_idx = right_stream.types().find(&right_name)?;
let output_type = get_output_type(left_stream.types(), right_stream.types(), right_idx)?;
let output = context.output.initialize(output_type)?;

3
src/lib/stream/mod.rs
View File

@ -45,11 +45,12 @@ pub fn declare(root: &Scope) -> CrushResult<()> {
sum_avg::Avg::declare(env)?;
sum_avg::Min::declare(env)?;
sum_avg::Max::declare(env)?;
sum_avg::Mul::declare(env)?;
env.declare_command(
"select", select::select, true,
"select copy_fields:field... [%] new_field=definition:command",
"Pass on some old fields and calculate new ones for each line of input",
example!(r#"ls | select ^user path={"{}/{}":format (pwd) file}"#), Unknown,
example!(r#"ls | select user path={"{}/{}":format (pwd) file}"#), Unknown,
vec![],
)?;
seq::Seq::declare(env)?;

2
src/lib/stream/reverse.rs
View File

@ -14,7 +14,7 @@ pub struct Reverse {
fn reverse(context: CommandContext) -> CrushResult<()> {
Reverse::parse(context.arguments.clone(), &context.global_state.printer())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let output = context.output.initialize(input.types().to_vec())?;
let mut q: Vec<Row> = Vec::new();

20
src/lib/stream/select.rs
View File

@ -1,7 +1,7 @@
use crate::lang::command::Command;
use crate::lang::errors::error;
use crate::lang::execution_context::CommandContext;
use crate::lang::pipe::{pipe, empty_channel, Stream};
use crate::lang::pipe::{pipe, Stream};
use crate::lang::data::table::ColumnVec;
use crate::{
lang::errors::argument_error_legacy,
@ -10,7 +10,7 @@ use crate::{
lang::{argument::Argument, data::table::Row, value::Value},
util::replace::Replace,
};
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
enum Action {
Replace(usize),
@ -55,7 +55,7 @@ pub fn run(config: Config, mut input: Stream, context: CommandContext) -> CrushR
Argument::named(cell_type.name.as_ref(), cell.clone(), config.location)
})
.collect();
closure.invoke(context.empty().with_output(sender))?;
closure.eval(context.empty().with_args(arguments, None).with_output(sender))?;
receiver.recv()?
}
Source::Argument(idx) => row.cells()[*idx].clone(),
@ -98,7 +98,7 @@ pub fn run(config: Config, mut input: Stream, context: CommandContext) -> CrushR
.map(|(cell, cell_type)| Argument::named(&cell_type.name, cell.clone(), config.location))
.collect();
let (sender, receiver) = pipe();
closure.invoke(context.empty().with_output(sender))?;
closure.eval(context.empty().with_args(arguments, None).with_output(sender))?;
receiver.recv()?
}
Source::Argument(idx) => row.cells()[*idx].clone(),
@ -118,7 +118,7 @@ pub fn run(config: Config, mut input: Stream, context: CommandContext) -> CrushR
}
pub fn select(mut context: CommandContext) -> CrushResult<()> {
match context.input.clone().recv()?.stream() {
match context.input.clone().recv()?.stream()? {
Some(input) => {
let mut copy = false;
let mut columns = Vec::new();
@ -142,7 +142,7 @@ pub fn select(mut context: CommandContext) -> CrushResult<()> {
location = location.union(a.location);
match (a.argument_type.as_deref(), a.value.clone()) {
(Some(name), Value::Command(closure)) => {
match (copy, input_type.find_str(name)) {
match (copy, input_type.find(name)) {
(true, Ok(idx)) => {
columns.push((Action::Replace(idx), Source::Closure(closure)))
}
@ -152,16 +152,16 @@ pub fn select(mut context: CommandContext) -> CrushResult<()> {
)),
}
}
(None, Value::Field(name)) => {
(None, Value::Symbol(name)) => {
if name.len() != 1 {
return argument_error_legacy("Invalid field");
}
match (copy, input_type.find_str(name[0].as_ref())) {
match (copy, input_type.find(name.as_ref())) {
(false, Ok(idx)) => columns
.push((Action::Append(name[0].clone()), Source::Argument(idx))),
.push((Action::Append(name.clone()), Source::Argument(idx))),
_ => {
return argument_error_legacy(
format!("Unknown field {}", name[0]).as_str(),
format!("Unknown field {}", name).as_str(),
);
}
}

5
src/lib/stream/seq.rs
View File

@ -1,3 +1,4 @@
use std::mem;
use crate::lang::errors::CrushResult;
use crate::lang::execution_context::CommandContext;
use crate::lang::data::table::ColumnType;
@ -22,9 +23,7 @@ pub fn seq(context: CommandContext) -> CrushResult<()> {
.initialize(vec![ColumnType::new("value", ValueType::Integer)])?;
if (cfg.to > cfg.from) != (cfg.step > 0) {
let tmp = cfg.to;
cfg.to = cfg.from;
cfg.from = tmp;
mem::swap(&mut cfg.to, &mut cfg.from);
}
let mut idx = cfg.from;

8
src/lib/stream/sort.rs
View File

@ -3,7 +3,7 @@ use crate::lang::errors::{error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::data::table::ColumnVec;
use crate::lang::data::table::Row;
use crate::lang::value::Field;
use crate::lang::value::Symbol;
use crate::lang::errors::argument_error_legacy;
use signature::signature;
use std::cmp::Ordering;
@ -11,19 +11,19 @@ use std::cmp::Ordering;
#[signature(
sort,
short = "Sort input based on column",
example = "ps | sort ^cpu",
example = "ps | sort cpu",
output = Passthrough)]
pub struct Sort {
#[unnamed()]
#[description("the columns to sort on. Optional if input only has one column.")]
field: Vec<Field>,
field: Vec<Symbol>,
#[description("reverse the sort order.")]
#[default(false)]
reverse: bool,
}
fn sort(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let output = context.output.initialize(input.types().to_vec())?;
let cfg: Sort = Sort::parse(context.arguments, &context.global_state.printer())?;

71
src/lib/stream/sum_avg.rs
View File

@ -7,9 +7,9 @@ use crate::lang::{value::Value, value::ValueType};
use chrono::Duration;
use float_ord::FloatOrd;
use signature::signature;
use crate::lang::value::Field;
use crate::lang::value::Symbol;
fn parse(input_type: &[ColumnType], field: Option<Field>) -> CrushResult<usize> {
fn parse(input_type: &[ColumnType], field: Option<String>) -> CrushResult<usize> {
field.map(|f| input_type.find(&f))
.unwrap_or_else(||
if input_type.len() == 1 {
@ -42,13 +42,13 @@ sum_function!(sum_duration, Duration, Duration::seconds(0), Duration);
#[signature(
sum,
short = "Calculate the sum for the specific column across all rows.",
example = "ps | sum ^cpu")]
example = "proc:list | sum cpu")]
pub struct Sum {
field: Option<Field>,
field: Option<Symbol>,
}
fn sum(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(input) => {
let cfg: Sum = Sum::parse(context.arguments, &context.global_state.printer())?;
let column = parse(input.types(), cfg.field)?;
@ -94,13 +94,13 @@ avg_function!(avg_duration, Duration, Duration::seconds(0), Duration, i32);
#[signature(
avg,
short = "Calculate the average for the specific column across all rows.",
example = "ps | avg ^cpu")]
example = "proc:list | avg cpu")]
pub struct Avg {
field: Option<Field>,
field: Option<Symbol>,
}
fn avg(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(input) => {
let cfg: Avg = Avg::parse(context.arguments, &context.global_state.printer())?;
let column = parse(input.types(), cfg.field)?;
@ -159,13 +159,13 @@ aggr_function!(max_time, Time, |a, b| std::cmp::max(a, b));
#[signature(
min,
short = "Calculate the minimum for the specific column across all rows.",
example = "ps | min ^cpu")]
example = "proc:list | min cpu")]
pub struct Min {
field: Option<Field>,
field: Option<Symbol>,
}
fn min(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(input) => {
let cfg: Min = Min::parse(context.arguments, &context.global_state.printer())?;
let column = parse(input.types(), cfg.field)?;
@ -186,13 +186,13 @@ fn min(context: CommandContext) -> CrushResult<()> {
#[signature(
max,
short = "Calculate the maximum for the specific column across all rows.",
example = "ps | max ^cpu")]
example = "proc:list | max cpu")]
pub struct Max {
field: Option<Field>,
field: Option<Symbol>,
}
fn max(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(input) => {
let cfg: Max = Max::parse(context.arguments, &context.global_state.printer())?;
let column = parse(input.types(), cfg.field)?;
@ -209,3 +209,46 @@ fn max(context: CommandContext) -> CrushResult<()> {
_ => error("Expected a stream"),
}
}
macro_rules! mul_function {
($name:ident, $var_type:ident, $var_initializer:expr, $value_type:ident) => {
fn $name(mut s: Stream, column: usize) -> CrushResult<Value> {
let mut res: $var_type = $var_initializer;
while let Ok(row) = s.read() {
match row.cells()[column] {
Value::$value_type(i) => res = res * i,
_ => return error("Invalid cell value"),
}
}
Ok(Value::$value_type(res))
}
};
}
mul_function!(mul_int, i128, 1, Integer);
mul_function!(mul_float, f64, 1.0, Float);
#[signature(
mul,
short = "Calculate the product for the specific column across all rows.",
example = "seq 5 10 | mul")]
pub struct Mul {
field: Option<Symbol>,
}
fn mul(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream()? {
Some(input) => {
let cfg: Sum = Sum::parse(context.arguments, &context.global_state.printer())?;
let column = parse(input.types(), cfg.field)?;
match &input.types()[column].cell_type {
ValueType::Integer => context.output.send(mul_int(input, column)?),
ValueType::Float => context.output.send(mul_float(input, column)?),
t => argument_error_legacy(
&format!("Can't calculate product of elements of type {}", t),
),
}
}
_ => error("Expected a stream"),
}
}

2
src/lib/stream/tail.rs
View File

@ -19,7 +19,7 @@ pub struct Tail {
fn tail(context: CommandContext) -> CrushResult<()> {
let cfg: Tail = Tail::parse(context.arguments, &context.global_state.printer())?;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let output = context.output.initialize(input.types().to_vec())?;
let mut q: VecDeque<Row> = VecDeque::new();

9
src/lib/stream/uniq.rs
View File

@ -2,23 +2,22 @@ use crate::lang::errors::{error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::data::table::ColumnVec;
use crate::lang::data::table::Row;
use crate::lang::value::Value;
use crate::lang::value::{Symbol, Value};
use std::collections::HashSet;
use signature::signature;
use crate::lang::command::OutputType::Passthrough;
use crate::lang::value::Field;
#[signature(
uniq,
output = Passthrough,
short = "Only output the first row if multiple rows has the same value for the specified column",
example = "ps | uniq ^user")]
example = "ps | uniq user")]
pub struct Uniq {
field: Option<Field>,
field: Option<Symbol>,
}
pub fn uniq(context: CommandContext) -> CrushResult<()> {
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let cfg: Uniq = Uniq::parse(context.arguments, &context.global_state.printer())?;
let output = context.output.initialize(input.types().to_vec())?;

8
src/lib/stream/where.rs
View File

@ -2,11 +2,11 @@ use crate::lang::command::Command;
use crate::lang::command::OutputType::Passthrough;
use crate::lang::errors::{error, CrushResult};
use crate::lang::execution_context::CommandContext;
use crate::lang::pipe::{black_hole, pipe, empty_channel};
use crate::lang::{argument::Argument, data::table::ColumnType};
use crate::lang::{data::table::Row, value::Value};
use signature::signature;
use crate::lang::ast::Location;
use crate::lang::ast::location::Location;
use crate::lang::pipe::pipe;
#[signature(
r#where,
@ -35,7 +35,7 @@ fn evaluate(
let (sender, reciever) = pipe();
condition.invoke(
condition.eval(
base_context
.clone()
.with_args(arguments, None)
@ -52,7 +52,7 @@ pub fn r#where(context: CommandContext) -> CrushResult<()> {
let cfg: Where = Where::parse(context.arguments.clone(), &context.global_state.printer())?;
let location = context.arguments[0].location;
match context.input.recv()?.stream() {
match context.input.recv()?.stream()? {
Some(mut input) => {
let base_context = context.empty();

7
src/lib/stream/zip.rs
View File

@ -3,7 +3,12 @@ use crate::lang::execution_context::CommandContext;
use crate::lang::pipe::Stream;
use signature::signature;
#[signature(zip, can_block = true, short = "Combine two streams of data into one")]
#[signature(
zip,
can_block = true,
short = "Combine two streams of data into one containing one row of each input stream in each row of output.",
long = "If the two streams have different numbers of rows, the longer stream will be truncated to the length\nof the shorter one."
)]
pub struct Zip {
#[description("the first stream.")]
first: Stream,

68
src/lib/types/float.rs
View File

@ -85,6 +85,46 @@ lazy_static! {
Known(ValueType::Bool),
vec![],
);
res.declare(
full("max"),
max,
false,
"float:max",
"Largest finite float value",
None,
Known(ValueType::Float),
vec![],
);
res.declare(
full("min"),
min,
false,
"float:min",
"Smallest finite float value",
None,
Known(ValueType::Float),
vec![],
);
res.declare(
full("nan"),
nan,
false,
"float:nan",
"Not a Number",
None,
Known(ValueType::Float),
vec![],
);
res.declare(
full("infinity"),
infinity,
false,
"float:infinity",
"Infinity",
None,
Known(ValueType::Float),
vec![],
);
res
};
}
@ -148,3 +188,31 @@ fn is_infinite(context: CommandContext) -> CrushResult<()> {
.output
.send(Value::Bool(context.this.float()?.is_infinite()))
}
fn max(context: CommandContext) -> CrushResult<()> {
context.arguments.check_len(0)?;
context
.output
.send(Value::Float(f64::MAX))
}
fn min(context: CommandContext) -> CrushResult<()> {
context.arguments.check_len(0)?;
context
.output
.send(Value::Float(f64::MIN))
}
fn nan(context: CommandContext) -> CrushResult<()> {
context.arguments.check_len(0)?;
context
.output
.send(Value::Float(f64::NAN))
}
fn infinity(context: CommandContext) -> CrushResult<()> {
context.arguments.check_len(0)?;
context
.output
.send(Value::Float(f64::INFINITY))
}

2
src/lib/types/list.rs
View File

@ -207,7 +207,7 @@ fn of(mut context: CommandContext) -> CrushResult<()> {
match context.arguments.len() {
0 => {
let mut lst = Vec::new();
let mut input = mandate(context.input.recv()?.stream(), "Expected a stream")?;
let mut input = mandate(context.input.recv()?.stream()?, "Expected a stream")?;
if input.types().len() != 1 {
return data_error("Expected input with exactly one column");
}

4
src/lib/types/mod.rs
View File

@ -48,7 +48,7 @@ fn new(mut context: CommandContext) -> CrushResult<()> {
if let Some(Value::Command(c)) = res.get("__init__") {
context.output = black_hole();
context.this = Some(Value::Struct(res.clone()));
c.invoke(context)?;
c.eval(context)?;
}
o.send(Value::Struct(res))
}
@ -219,7 +219,7 @@ pub fn declare(root: &Scope) -> CrushResult<()> {
env.declare("scope", Value::Type(ValueType::Scope))?;
env.declare("binary", Value::Type(ValueType::Binary))?;
env.declare("binary_stream", Value::Type(ValueType::BinaryInputStream))?;
env.declare("field", Value::Type(ValueType::Field))?;
env.declare("field", Value::Type(ValueType::Symbol))?;
env.declare("empty", Value::Type(ValueType::Empty))?;
env.declare("float", Value::Type(ValueType::Float))?;
env.declare("integer", Value::Type(ValueType::Integer))?;

2
src/lib/types/table_output_stream.rs
View File

@ -62,7 +62,7 @@ struct Write {}
fn write(context: CommandContext) -> CrushResult<()> {
context.output.send(Value::Empty())?;
let real_output = context.this.table_output_stream()?;
let mut stream = mandate(context.input.recv()?.stream(), "Expected a stream")?;
let mut stream = mandate(context.input.recv()?.stream()?, "Expected a stream")?;
while let Ok(row) = stream.read() {
real_output.send(row)?;

6
src/util/directory_lister.rs
View File

@ -1,3 +1,4 @@
use std::collections::VecDeque;
/**
A simple wrapper around std::fs::read_dir to allow for unit testing via fakes.
@ -9,10 +10,9 @@ you'll need something cleverer.
*/
use std::path::{PathBuf};
use crate::lang::errors::{CrushResult, to_crush_error, mandate};
use crate::lang::errors::{CrushResult, mandate, to_crush_error};
use std::fs::{ReadDir, read_dir};
use ordered_map::{OrderedMap, Entry};
use std::collections::VecDeque;
use ordered_map::{Entry, OrderedMap};
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct Directory {

4
src/util/escape.rs
View File

@ -1,6 +1,7 @@
use crate::util::hex::from_hex;
use crate::lang::errors::{CrushResult, to_crush_error, mandate, data_error};
use crate::lang::errors::{to_crush_error, data_error};
use std::convert::TryFrom;
use crate::CrushResult;
pub fn escape_without_quotes(s: &str) -> String {
let mut res = String::with_capacity(s.len());
@ -41,7 +42,6 @@ enum State {
pub fn unescape(s: &str) -> CrushResult<String> {
use State::*;
use crate::lang::errors::CrushResult;
let mut res = "".to_string();
let mut state = Normal;

4
tests/chmod.crush
View File

@ -1,4 +1,4 @@
touch ./foo
foo:chmod "a=" "o+xr" "u+w" "g-r"
find ./foo | select ^permissions
./foo:chmod "a=" "o+xr" "u+w" "g-r"
find ./foo | select permissions
rm ./foo

31
tests/class.crush
View File

@ -1,22 +1,23 @@
Point := (class)
Point:__init__ = {
|x:float y:float|
this:x = x
this:y = y
$Point := (class)
$Point:__init__ = {
|$x:$float $y:$float|
this:x = $x
this:y = $y
}
Point:len = {
$Point:len = {
||
math:sqrt this:x*this:x + this:y*this:y
math:sqrt $this:x*$this:x + $this:y*$this:y
}
Point:__add__ = {
|@unnamed|
other := unnamed[0]
Point:new x=(this:x + other:x) y=(this:y + other:y)
$Point:__add__ = {
|@ $unnamed|
$other := $unnamed[0]
Point:new x=($this:x + $other:x) y=($this:y + $other:y)
}
p1 := (Point:new x=0.0 y=4.0)
p2 := (Point:new x=3.0 y=0.0)
p3 := p1 + p2
p3:len
$p1 := (Point:new x=0.0 y=4.0)
$p2 := (Point:new x=3.0 y=0.0)
$p3 := $p1 + $p2
$p3:len

6
tests/closure_signatures.crush
View File

@ -1,3 +1,3 @@
ggg := {|a : (dict integer integer)| echo a}
hhh := ((dict integer integer):new)
ggg a=hhh
$ggg := {|$a : (dict $integer $integer)| echo $a}
$hhh := ((dict $integer $integer):new)
ggg a=$hhh

26
tests/comparisons.crush
View File

@ -1,4 +1,4 @@
echo "Integer comparisons"
"Integer comparisons"
1>2
1<2
@ -21,7 +21,7 @@ echo "Integer comparisons"
2!=1
2==1
echo "Float comparisons"
"Float comparisons"
1.0>2.0
1.0<2.0
@ -30,7 +30,7 @@ echo "Float comparisons"
1.0!=2.0
1.0==2.0
echo "Duration comparisons"
"Duration comparisons"
(duration:of seconds=1) > (duration:of seconds=2)
(duration:of seconds=1) < (duration:of seconds=2)
@ -39,18 +39,18 @@ echo "Duration comparisons"
(duration:of seconds=1) != (duration:of seconds=2)
(duration:of seconds=1) == (duration:of seconds=2)
echo "Time comparisons"
"Time comparisons"
t1 := (time:now)
t2 := (time:now + (duration:of seconds=1))
t1 > t2
t1 < t2
t1 >= t2
t1 <= t2
t1 != t2
t1 == t2
$t1 := (time:now)
$t2 := (time:now + (duration:of seconds=1))
$t1 > $t2
$t1 < $t2
$t1 >= $t2
$t1 <= $t2
$t1 != $t2
$t1 == $t2
echo "String comparisons"
"String comparisons"
"a" > "b"
"a" < "b"
"a" >= "b"

8
tests/convert.crush
View File

@ -1,4 +1,4 @@
convert 1.0 integer
convert 0.5 integer
convert 5 string
typeof (convert 5 string)
convert 1.0 $integer
convert 0.5 $integer
convert 5 $string
typeof (convert 5 $string)

2
tests/find.crush
View File

@ -1 +1 @@
find example_data/tree | select ^file | sort ^file
find ./example_data/tree | select file | sort file

28
tests/for.crush
View File

@ -1,33 +1,33 @@
for i=(list:of 1 2) {
echo i
echo $i
}
d := ((dict string integer):new)
d["fooo"] = 3
$d := ((dict $string $integer):new)
$d["fooo"] = 3
for d {
echo key value
for $d {
echo $key $value
}
for i=d {
echo i:key i:value
for i=$d {
echo $i:key $i:value
}
table := (seq 3 | materialize)
$table := (seq 3 | materialize)
for table {
echo value
for $table {
echo $value
}
for i=table {
echo i
for i=$table {
echo $i
}
for (seq 3) {
echo value
echo $value
}
for i=(seq 3) {
echo i
echo $i
}

6
tests/group.crush
View File

@ -1,3 +1,3 @@
seq 1000 | select num={(value*5):mod 7} | group ^num | sort ^num
seq 1000 | select num={(value*5):mod 7} | group ^num c={count} | sort ^num
seq 1000 | select num={(value*5):mod 7} | group ^num c={count} s={sum} | sort ^num
seq 10 | select num={($value*5):mod 7} | group num #| sort num
#seq 1000 | select num={($value*5):mod 7} | group num c={count} | sort num
#seq 1000 | select num={($value*5):mod 7} | group num c={count} s={sum} | sort num

6
tests/join.crush
View File

@ -1,4 +1,4 @@
home:=(csv:from example_data/home.csv name=string country=string)
age:=(csv:from example_data/age.csv name=string age=integer)
$home:=(csv:from ./example_data/home.csv name=$string country=$string)
$age:=(csv:from ./example_data/age.csv name=$string age=$integer)
join name=home name=age | sort ^name
join name=$home name=$age | sort name

6
tests/json.crush
View File

@ -1,8 +1,8 @@
json:from example_data/din%.json |
sort ^name
sort name
json:from example_data/din%.json |
where {name =~ re"Tri.*"}
where {$name =~ re"Tri.*"}
# Check serialisation and deserialisation, including field order
seq 5|select ^value half={0.5 * value} str={convert value string} struct={data value=value} | json:to |json:from
seq 5|select value half={0.5 * $value} str={convert $value $string} struct={data value=$value} | json:to |json:from

6
tests/loop.crush
View File

@ -4,10 +4,10 @@ loop {
echo "NO"
}
a := false
$a := $false
loop {
if a break
a = true
if $a $break
$a = $true
echo 2
continue
echo "NO"

14
tests/pipe.crush
View File

@ -2,20 +2,20 @@
# of multiple commands into the same pipe.
# Create the pipe
pipe := ((table_input_stream value=integer):pipe)
$pipe := ((table_input_stream value=$integer):pipe)
# Attach the input end to a bunch of seq commands
_1 := (seq 100_000 | pipe:output:write | bg)
_2 := (seq 100_000 | pipe:output:write | bg)
_3 := (seq 100_000 | pipe:output:write | bg)
_4 := (seq 100_000 | pipe:output:write | bg)
$_1 := (seq 100_000 | pipe:output:write | bg)
$_2 := (seq 100_000 | pipe:output:write | bg)
$_3 := (seq 100_000 | pipe:output:write | bg)
$_4 := (seq 100_000 | pipe:output:write | bg)
# Attach the output end to the sum command and
sum_job_id := (pipe:input | sum | bg)
$sum_job_id := (pipe:input | sum | bg)
# Close the output and input, so that the input can actually
# reach EOF once the above 4 invocation exit
pipe:close
# Wait for the sum command to finish
sum_job_id | fg
$sum_job_id | fg

14
tests/pipe2.crush
View File

@ -2,19 +2,19 @@
# of a command between multiple commands via the same pipe.
# Create the pipe
pipe := ((table_input_stream value=integer):pipe)
$pipe := ((table_input_stream value=$integer):pipe)
# Attach the input end to a bunch of seq commands
input := (seq 10_000 | pipe:output:write | bg)
$input := (seq 10_000 | pipe:output:write | bg)
# Attach the output end to the sum command and
sum_job_id1 := (pipe:input | sum | bg)
sum_job_id2 := (pipe:input | sum | bg)
sum_job_id3 := (pipe:input | sum | bg)
sum_job_id4 := (pipe:input | sum | bg)
$sum_job_id1 := (pipe:input | sum | bg)
$sum_job_id2 := (pipe:input | sum | bg)
$sum_job_id3 := (pipe:input | sum | bg)
$sum_job_id4 := (pipe:input | sum | bg)
# Close the output and input, so that the input can actually
# reach EOF once the above 4 invocation exit
pipe:close
# Wait for the sum command to finish
val (sum_job_id1 | fg) + (sum_job_id2 | fg) + (sum_job_id3 | fg) + (sum_job_id4 | fg)
val ($sum_job_id1 | fg) + ($sum_job_id2 | fg) + ($sum_job_id3 | fg) + ($sum_job_id4 | fg)

20
tests/pipe3.crush
View File

@ -2,23 +2,23 @@
# can use a single pipe both for input and output.
# Create the pipe
pipe := ((table_input_stream value=integer):pipe)
$pipe := ((table_input_stream value=$integer):pipe)
# Attach the input end to a bunch of seq commands
input1 := (seq 100_000 | pipe:output:write | bg)
input2 := (seq 100_000 | pipe:output:write | bg)
input3 := (seq 100_000 | pipe:output:write | bg)
input4 := (seq 100_000 | pipe:output:write | bg)
$input1 := (seq 100_000 | pipe:output:write | bg)
$input2 := (seq 100_000 | pipe:output:write | bg)
$input3 := (seq 100_000 | pipe:output:write | bg)
$input4 := (seq 100_000 | pipe:output:write | bg)
# Attach the output end to the sum command and
sum_job_id1 := (pipe:input | sum | bg)
sum_job_id2 := (pipe:input | sum | bg)
sum_job_id3 := (pipe:input | sum | bg)
sum_job_id4 := (pipe:input | sum | bg)
$sum_job_id1 := (pipe:input | sum | bg)
$sum_job_id2 := (pipe:input | sum | bg)
$sum_job_id3 := (pipe:input | sum | bg)
$sum_job_id4 := (pipe:input | sum | bg)
# Close the output and input, so that the input can actually
# reach EOF once the above invocations exit
pipe:close
# Wait for the sum commands to finish
val (sum_job_id1 | fg) + (sum_job_id2 | fg) + (sum_job_id3 | fg) + (sum_job_id4 | fg)
val ($sum_job_id1 | fg) + ($sum_job_id2 | fg) + ($sum_job_id3 | fg) + ($sum_job_id4 | fg)

10
tests/serialization.crush
View File

@ -1,15 +1,15 @@
rm ./.test_file
# Pipe output of find into a file
find example_data/tree|select ^file ^type | pup:to ./.test_file
find ./example_data/tree|select file type | sort file | pup:to ./.test_file
# And read it back out again
pup:from ./.test_file
# Create a closure that close on outside variables
a := 4
b := 7.5
fff := {|c:integer=1 d| echo a*b*c; for (seq 3) d }
$a := 4
$b := 7.5
$fff := {|$c:$integer=1 $d| echo $a*$b*$c; for (seq 3) $d }
# Serialize the closure
val fff | pup:to ./.test_file
val $fff | pup:to ./.test_file
# Unset the variables used by the closure
var:unset "a" "b" "fff"
# Deserialize the closure and check that the variables still exists.

Some files were not shown because too many files have changed in this diff Show More