* docs: implemented python script to convert local documentation from .md files to .html files
* docs: filled index.md; reviewed all references in .md files
* docs: updated Embedded.md and resized carp_on_arduboy.jpg to sensible width
* docs: copy sub folders (./docs/core and ./docs/sdl) to ./docs-html/ and refer docu to it
* docs: phrased requirements more clearly
* docs: generate docs for Standard libraries before converting .md docs to .html
* docs: change index to markdown lists
* docs: index.md worked around limitation in md converter
* docs: removed modules count from Libraries.md
Co-authored-by: guberatsie <gunnar.bernhardt@siemens.com>
* chore: Abuse deftemplate to get rid of Address
* chore: Avoid semicolon at end of define
* fix: address should be useable as an argument to a HOF
* chore: adapt examples to new address signature
* fix: Remove Address from typeOf
* fix: Remove more uses of Address
* fix: Remove more mentions of address in the Haskell code
* fix: Remove test that ensure you can't take the address of non-symbols
* refactor: Moved `address` to Pointer.carp
* refactor: Move address into Pointer module
Co-authored-by: Jorge Acereda <jacereda@gmail.com>
* chore: Re-format Haskell code
* fix: Unify aupdate and aupdate! with other update functions
* fix: Re-add comment
* fix: Also make StaticArray.aupdate! adhere to the normal update signature
* fix: Failing test
* test: Test for error when recursing using wrong nr of args
* test: Wrong type when recursing
* test: Defining function with def
* test: Using special symbol as binder
* test: Dynamic closures can refer to itself
* test: Avoid unification failure
* fix: Address feedback
This fix is a close cousin of the one that allowed let bindings to
shadow global commands. We now allow function arguments to shadow
commands as well by using a local lookup preference for argument names,
making functions such as:
```
(defndynamic foo [car]
(Symbol.prefix car 'foo)
```
work as anticipated. I've also removed unused code from `apply`.
Fixes#1057
* fix: don't expand inner module macros on first pass; privacy
This commit changes the behavior of expansions to avoid expanding module
expressions until we're actually processing the module in question.
Previously, the following form would be entirely expanded at the time of evaluating A:
```clojure
(defmodule A <- current environment
(some-macro) <- expand
(defmodule B
(some-macro f) <- expand, current env is A, *NOT* B.
so if this expands to
(private f)
(defn f ....)
the f of the expansion is added to *A*, and we have a duplicate
ghost binder.
)
(defn foo [] B.f) <- expand, B.f does not exist yet, any meta on the
binding will be ignored, permitting privacy errors since expansion
ignores undefined bindings, instead, we'll look this up at eval time,
and not check privacy as doing so would cause problems for legitimate
cases.
)
```
This meant that if the macro happened to have side-effects, e.g. calling
`meta-set!` we'd produce side-effects in the wrong environment, A,
resulting in duplicate bindings, missing bindings at evaluation time,
and other problems.
Now, we instead process the form as follows:
```clojure
(defmodule A <- current environment
(some-macro) <- expand
(defmodule B
(some-macro f) <- wait
)
(defn foo [] B.f)
)
;; step 2
(defmodule A
(foo-bar ) <- previously expanded macro
(defmodule B <- current environment
(some-macro f) <- expand
)
....
)
```
In general, this prevents the generation of a bunch of unintentional and
incorrectly added bindings when calling `meta-set!` from various macros.
Additionally, privacy constraints are now carried across nested modules:
```
(defmodule A
(defmodule B
(private f)
(defn f [] 0)
)
(defn g [] (B.f)) ;; Privacy error!
)
```
This change also fixed an issue whereby recursive functions with `sig`
annotations could trick the compiler. Again, this had to do with the
unintentionally added bindings stemming from expansion of nested module
expressions via meta-set.
Fixes#1213, Fixes#467
* fix: ensure we check privacy against the path of found binders
* fix: don't shadow local bindings with dynamics
This commit adds a new lookup preference to the evaluator, LookupLocal,
and uses it to lookup bindings in the scope of let forms. This fixes an
issue whereby our original bias toward dynamic bindings would prevent
users from shadowing dynamic bindings with local bindings of the same
name. Before the following code returned `command c`:
```
(defdynamic test-val (let [c (car (list 1 2 3))]
c))
```
It now returns `1`.
I also fixed a small issue with top-level (as in, without a
corresponding function environment) let forms (they'd cause a crash for
lack of an environment parent).
Fixes#659
* refactor: only prefer local lookups for shadows
The prior commit introduced a local lookup preference into the evaluator
in order allow for shadowing of global names in local scopes (like let
bindings). However, this introduced prohibitive performance costs,
especially for dynamic functions.
To mitigate this, we only perform local-biased lookups for a set of
known-shadows. Since we know the names of local bindings at form
evaluation time, we can restrict our local lookup bias to these paths
only. This greatly reduces the performance penalties initially incurred
by the change.
I've also refactored some of the lookup code for clarity.
* fix: support recursive let bindings
Previously, the bodies of anonymous functions bound to a let name did
not have access to their names, making recursion such as:
```
(let [f (fn [x] (if (= x 1) x (f (dec x))))] (f 10))
```
impossible. We now equip evaluation of let bindings with an additional
recursion environment making this possible. The example above will now
resolve to `1`.
Fixes#1133
Most def forms bind values, so their types are wholly determined at
initial assignment time. However, defs that bind to lambdas may have
polymorphic types by virtue of their use of interfaces:
```
(defn duplicate-arg [f]
(fn [x] (f x x)))
(def double (duplicate-arg +))
(defn main []
(println* (double 3)))
```
Previously, the above program would yield a concretization error. This
commit adds concretization handling for def forms, and makes it possible
to call lambdas bound to defs as is done in the program above.
(Note: the example will emit an error when run as it results in main
returning an error code, 6, but the program works as intended.)
Fixes#364
This commit fixes an issue whereby all recursive calls were given var
types, this resulted in strange behavior such as the following code:
```clojure
(defn recurse [a b]
(let [c (+ b 1)
out (recurse c)]
(+ out 1)))
(defn main []
(println* (recurse 1 2)))
```
compiling just fine and yielding runtime segfaults. Now, if a recursive
instance of a symbol was previously typed, we use that type--if not, we
assign a fresh type variable (the old behavior). This fixes code like
that above, throwing an error at compile time about an incorrect number
of arguments for `recurse`.
Fixes#348
Our use of type variables in deftype forms to support parametric
polymorphism relied on precise matching between variables in the head
and the body of the form. However, this could lead to unification
failure when the concretizer encountered a type that mixed two
polymorphic types that use the same type var name:
```
(deftype (HitRecord a) [ t a ])
(deftype (CurrentHit a) [ hr (Maybe (HitRecord a)) ])
```
Concretizing this would previously attempt to bind both Maybe(HitRecord
) and just (HitRecord) to `a`.
To fix this, we temporarily rename all type variables in type
definitions during concretization. Names are preserved in place, so
there's no danger of losing a variable. The code above will now work:
```
=> (CurrentHit.init (Just (HitRecord.init 5)))
(CurrentHit (Just (HitRecord 5)))
=> (HitRecord.init 3)
(HitRecord 3)
```
Fixes#521
* refactor: major environment mgmt refactor
This big refactor primarily changes two things in terms of behavior:
1. Stores a SymPath on concretely named (non-generic) struct types;
before we stored a string.
2. The SymPath mentioned in (1.) designates where the struct is stored
in the current environment chain. Modules now carry a local type
environment in addition to their local value environments. Any types
defined in the module are added to this environment rather than the
global type environment.
To resolve a type such as `Foo.Bar` we now do the following:
- Search the *global value environment* for the Foo module.
- Get the type environment stored in the Foo module.
- Search for Bar in the Foo module's type environment.
Additionally, this commit eliminates the Lookup module entirely and
refactors the Env module to handle all aspects of environment management
in hopefully a more reusable fashion.
I also took the opportunity to refactor primitiveDeftype in Primitives
and qualifySym in Qualify, both of which were hefty functions that I
found difficult to grok and needed refactoring anyway as a result of
lookup changes (lookups now return an Either instead of a Maybe).
Subsequent commits will clean up and clarify this work further.
This does include one minor regression. Namely, an implementation of
`hash` in core/Color that was maximally generic now needs type casting.
* refactor: clean up recent Env changes
This commit removes some redundant functions, unifies some logic, and
renames some routines across the Env module in efforts to make it
cleaner. Call sites have been updated accordingly.
* chore: format code with ormolu
* fix: update lookup tests
Changes references to renamed functions in the Env module.
* refactor: style + additional improvements from eriksvedang@
- Rename arrayTy -> arrayTyA in ArrayTemplates.hs to disambiguate.
- Add maybeId util function.
- Remove commented code.
- Refactor a few functions for readability.
* fix: fix type inference regression
Recent commits introduced one minor regression whereby an instance of
type inference in core/Color.carp no longer worked and required
explicit type annotation. The problem ultimately had to do with
qualification:
- Prior to the recent changes, type inference worked because the call in
question was qualified to Color.Id.get-tag, fixing the type.
- Failing to copy over a local envs Use modules to function envs
resulted in finding more than just Color.Id.get-tag for this instance.
We now copy use modules over to function envs generated during
qualification to ensure we resolve to Use'd definitions before more
general cases.
Similarly, I made a small change to primitiveUse to support contextual
use calls (e.g. the `(use Id)` in Color.carp, which really means `(use
Color.Id)`)
* chore: Update some clarificatory comments
* chore: fix inline comment
* feat: add 'when' functions for Maybe and Result
These functions enable users to execute some side-effecting function
(one that returns unit and take no arguments) based on the contents of a
Maybe or Result.
`when-success`: Executes a side-effect when given a `Result.Success`
`when-error`: Executes a side-effect when given a `Result.Error`
`when-just`: Executes a side-effect when given a `Maybe.Just`
`when-nothing`: Executes a side-effect when given a `Maybe.Nothing`
* docs: add Control to core lib doc generation
* feat: Add support for emitting literal C
This commit adds a new Obj type, C, as well as a command for emitting C
literals (which are represented by this object).
This is necessary for circumstances in which Carp's handling of
templates and emission orders prevents interop with C. For example, the
c11 macro static_assert must only take a string literal in its second
position. However, Carp's memory management will typically assign a
string literal (in Carp) to an anonymous variable, which makes them
impossible to use with static_assert. The new emit-c command will
support this use case:
```
(static-assert 0 (Unsafe.emit-c "\"message!\""))
```
The literal string "message!" will be emitted in the compiler's C output
in the position corresponding to the macro call.
We also add a special type for c literals, CTy, to prevent conflating
them with Strings. This helps maintainers define clear boundaries and
express what interop code requires the use of literal C.
Likewise, we need to emit `c_code` to represent this type in C. This
wouldn't be necessary except that Carp sometimes auto-generates
functions that refer to Carp types in their C equivalents, so we need
this for completeness. It is typed as the void pointer.
N.B. That the command is not yet exposed in Carp in this commit.
Thanks to @TimDeve for recommending the name emit-c!
* feat: Add preproc command and add emit-c to Unsafe module
This commit adds the preproc command, which enables users to emit
arbitrary C in the compilers emitted C output.
The C emitted by calls to preproc will be appended to the output after
include directives but prior to any other emissions. One can use it to
call preprocessor directives in C, define helper functions etc.
preproc takes a C type value as an argument and must be used in
conjunction with emit-c. Both functions are added to the Unsafe module
to signal their dangerousness.
One can use `register` in combination with preproc to define code
entirely in Carp and obviate the need of additional header files. For
example:
```
(Unsafe.preproc (Unsafe.emit-c "#define FOO 0"))
(Unsafe.preproc (Unsafe.emit-c "void foo() { printf(\"%d\\n\", 1); }"))
(register FOO Int)
(register foo (Fn [] ()))
(defn main []
(do (foo)
(IO.println &(fmt "%d" FOO))))
```
The prior example emits C that defines a FOO macro and foo function,
which are both referenced in the main function emitted by Carps normal
processing. Here's what the output looks like:
```
// .. several other includes emitted by the compiler...
void foo() { printf("%d\n", 1); }
//Types:
// Depth 3
typedef struct {
union {
struct {
Long member0;
} Just;
// Nothing
char __dummy;
} u;
char _tag;
} Maybe__Long;
```
The C passed to preproc calls is emitted prior to Carps other emissions,
ensuring the user has access to these definitions before any Carp code
is called.
* docs: Add documentation on emit-c and preproc
* feat: add macros for emitting C compiler directives
This suite of macros uses the `Unsafe.emit-c` and `Unsafe.preproc`
functions to provide macros for emitting common C compiler directives,
such as #ifdef, #define, #pragma and others.
* refactor: Groups Dynamic together in Macros.carp
* fix: Fixes doc for `hidden` referring to the wrong symbol
* feat: Adds defn- & def- macros
Adding these macros as a shortand for declaring a def or defn and making
them `hidden` and `private`, useful to keep things internal to a module.
* test: Adds expected error output tests for def- & defn-
* refactor: Changes position of Module and Interface section in LanguageGuide
Trying to introduce concepts in the same order they are referred to in
the examples: structs > modules > interfaces.
* docs: Adds private & hidden section in the LanguageGuide
* refactor: move Context updates into functions
Previously, we had a lot of instances of updating Context records
directly, replacing environments where needed. This commit replaces
those hand-written record setting instances with functions, which should
allow us to more gracefully abstract over any preprocessing we should
have to do and help ensure we're updating contexts in the right way.
* refactor: replace inline context manipulation in primitives
Like the commit that altered Eval before it, this commit leverages
Context functions to remove a bunch of inline record field setting code
and direct env manipulation.
* refactor: replace generic binder lookups with contextual ones
* refactor: move true and false XObjs into Obj.hs
Previously, trueXObj and falseXObj were defined in Commands.hs, but
since they're just literal constructed XObj values, I feel Obj.hs is a
more appropriate home for them and makes them more widely accessible to
other modules without needing to import Commands.
* refactor: model symbol qualification requirements at typelevel
This commit refactors the Qualify module to express symbol qualification
at type level. In the past, all functions operated on SymPaths. In some
cases, the functions operated on paths that *were not yet qualified* and
so the functions would perform qualification inline. Contrarily, other
functions like define received XObjs (from another part of the codebase
entirely!) that were already fully qualified, and so it would be a grave
mistake to re-qualify them.
In the general case, it's difficult or impossible to tell across modules
whether or not a given SymPath is coming in qualified or unqualified,
which can easily lead to mistakes of double-qualification, e.g.
transforming `Foo.bar` into `Foo.Foo.bar`.
Modelling qualification in the type system enables us to avoid the
problem by distinguishing between unqualified and qualified paths. A
function receiving an SymPath can safely qualify it, whereas a function
receiving a QualifiedPath should not further qualify the path. This
helps better express and ensure constraints across modules.
In addition, this commit also refactors a few functions where there was
opportunity to do so.
* refactor: remove eval call from `doc`
This can lead to problems where a doc call intended to be evaluated
later (in a macro body) is evaluated *immediately* resulting in a
binding being added to the wrong scope (see the function reverse in
core).
The reason this behavior crops up now is that a special case for
evaluating module contexts was removed last commit--this special case
caused problems of its own, and the real root of things stems from the
unnecessary eval call. Generally, evaling a doc call provides no benefit
other than making evaluation of the meta set immediate in the repl,
which is easy enough for one to do on one's own by calling eval where
needed.
* refactor: use do notation to clarify case qualification
* refactor: rename runQualified to unQualified
@eriksvedang pointed out the `run` prefix typically denotes a monad. As
`Qualified` is not monadic (no monad instance defined) we drop the `r`
to ensure we don't mislead readers.
* refactor: convert a few more binds to do notation
Do notation is generally clearer in cases where we use mapM, etc. We can
also leverage liftM frequently in the qualification functions to
transform a Qualified xobj back into an xobj for further use.
* refactor: temporarily restore special case in meta set
Meta set disallows setting the meta of a prefixed, absolute path such as
`Foo.bar`. It only allows relative, unqualified paths `bar` and uses the
current context to determine the appropriate module.
If we eventually throw and error from envInsertAt, we can remove this
special case. I intend to do that later, but for now we'll keep the
special case to make for a more pleasant user experience.
