* refactor: refactor concretize module
This commit primarily refactors the concretize module, breaking out the
local definitions for visit functions into top level functions that are
hopefully easier to change. I've also modified some monadic code in the
interest of terseness.
This commit adds some additional patterns for XObj forms as well.
* refactor: Only export called functions in Concretize
Adds an export list to Concretize so that the module encapsulates those
functions that are only used internally within the module.
* refactor: better names in concretize functions
Clarify the names of variables in visitor type functions.
* refactor: Mid-refactor save point.
* feat: Code compiles
* refactor: Remove unused imports
* refactor: Move functions out of massive `manageMemory` block
* refactor: Move out even more functions from `manageMemory`
* refactor: Made most patterns match on "head form" of each s-expression
e.g. (if a b c) matches on 'if', 'a', 'b' and 'c'
* refactor: Use the pattern synonyms in Memory
* refactor: Remove a little cruft
* refactor: whenOK function
* refactor: Use 'whenRight' functions to avoid directly matching on Either
* docs: Comment the 'getConcretizedPath' function
* refactor: Move functionFinding-functions into Polymorphism module
The previous pattern matching changes introduced a subtle error in the
evaluator's handling of function applications, we used the same Resolver
for both lists and symbols, where previously we hadn't. Restoring the
old resolution selection and adding a new clause to capture forms like
((defn f [] 2)) restores the correct functioning in the repl.
Importantly, forms such as ((defn foo [x] x) 3), should only be resolved
when the defn form is the result of expanding a symbol.
Eventually, resolving defns to anonymous functions might simplify this
case.
Fixes#1237
* chore: Abuse deftemplate to get rid of Address
* chore: Avoid semicolon at end of define
* fix: address should be useable as an argument to a HOF
* chore: adapt examples to new address signature
* fix: Remove Address from typeOf
* fix: Remove more uses of Address
* fix: Remove more mentions of address in the Haskell code
* fix: Remove test that ensure you can't take the address of non-symbols
* refactor: Moved `address` to Pointer.carp
* refactor: Move address into Pointer module
Co-authored-by: Jorge Acereda <jacereda@gmail.com>
This fix is a close cousin of the one that allowed let bindings to
shadow global commands. We now allow function arguments to shadow
commands as well by using a local lookup preference for argument names,
making functions such as:
```
(defndynamic foo [car]
(Symbol.prefix car 'foo)
```
work as anticipated. I've also removed unused code from `apply`.
Fixes#1057
* fix: don't expand inner module macros on first pass; privacy
This commit changes the behavior of expansions to avoid expanding module
expressions until we're actually processing the module in question.
Previously, the following form would be entirely expanded at the time of evaluating A:
```clojure
(defmodule A <- current environment
(some-macro) <- expand
(defmodule B
(some-macro f) <- expand, current env is A, *NOT* B.
so if this expands to
(private f)
(defn f ....)
the f of the expansion is added to *A*, and we have a duplicate
ghost binder.
)
(defn foo [] B.f) <- expand, B.f does not exist yet, any meta on the
binding will be ignored, permitting privacy errors since expansion
ignores undefined bindings, instead, we'll look this up at eval time,
and not check privacy as doing so would cause problems for legitimate
cases.
)
```
This meant that if the macro happened to have side-effects, e.g. calling
`meta-set!` we'd produce side-effects in the wrong environment, A,
resulting in duplicate bindings, missing bindings at evaluation time,
and other problems.
Now, we instead process the form as follows:
```clojure
(defmodule A <- current environment
(some-macro) <- expand
(defmodule B
(some-macro f) <- wait
)
(defn foo [] B.f)
)
;; step 2
(defmodule A
(foo-bar ) <- previously expanded macro
(defmodule B <- current environment
(some-macro f) <- expand
)
....
)
```
In general, this prevents the generation of a bunch of unintentional and
incorrectly added bindings when calling `meta-set!` from various macros.
Additionally, privacy constraints are now carried across nested modules:
```
(defmodule A
(defmodule B
(private f)
(defn f [] 0)
)
(defn g [] (B.f)) ;; Privacy error!
)
```
This change also fixed an issue whereby recursive functions with `sig`
annotations could trick the compiler. Again, this had to do with the
unintentionally added bindings stemming from expansion of nested module
expressions via meta-set.
Fixes#1213, Fixes#467
* fix: ensure we check privacy against the path of found binders
* fix: don't shadow local bindings with dynamics
This commit adds a new lookup preference to the evaluator, LookupLocal,
and uses it to lookup bindings in the scope of let forms. This fixes an
issue whereby our original bias toward dynamic bindings would prevent
users from shadowing dynamic bindings with local bindings of the same
name. Before the following code returned `command c`:
```
(defdynamic test-val (let [c (car (list 1 2 3))]
c))
```
It now returns `1`.
I also fixed a small issue with top-level (as in, without a
corresponding function environment) let forms (they'd cause a crash for
lack of an environment parent).
Fixes#659
* refactor: only prefer local lookups for shadows
The prior commit introduced a local lookup preference into the evaluator
in order allow for shadowing of global names in local scopes (like let
bindings). However, this introduced prohibitive performance costs,
especially for dynamic functions.
To mitigate this, we only perform local-biased lookups for a set of
known-shadows. Since we know the names of local bindings at form
evaluation time, we can restrict our local lookup bias to these paths
only. This greatly reduces the performance penalties initially incurred
by the change.
I've also refactored some of the lookup code for clarity.
* fix: support recursive let bindings
Previously, the bodies of anonymous functions bound to a let name did
not have access to their names, making recursion such as:
```
(let [f (fn [x] (if (= x 1) x (f (dec x))))] (f 10))
```
impossible. We now equip evaluation of let bindings with an additional
recursion environment making this possible. The example above will now
resolve to `1`.
Fixes#1133
* refactor: major environment mgmt refactor
This big refactor primarily changes two things in terms of behavior:
1. Stores a SymPath on concretely named (non-generic) struct types;
before we stored a string.
2. The SymPath mentioned in (1.) designates where the struct is stored
in the current environment chain. Modules now carry a local type
environment in addition to their local value environments. Any types
defined in the module are added to this environment rather than the
global type environment.
To resolve a type such as `Foo.Bar` we now do the following:
- Search the *global value environment* for the Foo module.
- Get the type environment stored in the Foo module.
- Search for Bar in the Foo module's type environment.
Additionally, this commit eliminates the Lookup module entirely and
refactors the Env module to handle all aspects of environment management
in hopefully a more reusable fashion.
I also took the opportunity to refactor primitiveDeftype in Primitives
and qualifySym in Qualify, both of which were hefty functions that I
found difficult to grok and needed refactoring anyway as a result of
lookup changes (lookups now return an Either instead of a Maybe).
Subsequent commits will clean up and clarify this work further.
This does include one minor regression. Namely, an implementation of
`hash` in core/Color that was maximally generic now needs type casting.
* refactor: clean up recent Env changes
This commit removes some redundant functions, unifies some logic, and
renames some routines across the Env module in efforts to make it
cleaner. Call sites have been updated accordingly.
* chore: format code with ormolu
* fix: update lookup tests
Changes references to renamed functions in the Env module.
* refactor: style + additional improvements from eriksvedang@
- Rename arrayTy -> arrayTyA in ArrayTemplates.hs to disambiguate.
- Add maybeId util function.
- Remove commented code.
- Refactor a few functions for readability.
* fix: fix type inference regression
Recent commits introduced one minor regression whereby an instance of
type inference in core/Color.carp no longer worked and required
explicit type annotation. The problem ultimately had to do with
qualification:
- Prior to the recent changes, type inference worked because the call in
question was qualified to Color.Id.get-tag, fixing the type.
- Failing to copy over a local envs Use modules to function envs
resulted in finding more than just Color.Id.get-tag for this instance.
We now copy use modules over to function envs generated during
qualification to ensure we resolve to Use'd definitions before more
general cases.
Similarly, I made a small change to primitiveUse to support contextual
use calls (e.g. the `(use Id)` in Color.carp, which really means `(use
Color.Id)`)
* chore: Update some clarificatory comments
* chore: fix inline comment
* refactor: move Context updates into functions
Previously, we had a lot of instances of updating Context records
directly, replacing environments where needed. This commit replaces
those hand-written record setting instances with functions, which should
allow us to more gracefully abstract over any preprocessing we should
have to do and help ensure we're updating contexts in the right way.
* refactor: replace inline context manipulation in primitives
Like the commit that altered Eval before it, this commit leverages
Context functions to remove a bunch of inline record field setting code
and direct env manipulation.
* refactor: replace generic binder lookups with contextual ones
* refactor: move true and false XObjs into Obj.hs
Previously, trueXObj and falseXObj were defined in Commands.hs, but
since they're just literal constructed XObj values, I feel Obj.hs is a
more appropriate home for them and makes them more widely accessible to
other modules without needing to import Commands.
* refactor: model symbol qualification requirements at typelevel
This commit refactors the Qualify module to express symbol qualification
at type level. In the past, all functions operated on SymPaths. In some
cases, the functions operated on paths that *were not yet qualified* and
so the functions would perform qualification inline. Contrarily, other
functions like define received XObjs (from another part of the codebase
entirely!) that were already fully qualified, and so it would be a grave
mistake to re-qualify them.
In the general case, it's difficult or impossible to tell across modules
whether or not a given SymPath is coming in qualified or unqualified,
which can easily lead to mistakes of double-qualification, e.g.
transforming `Foo.bar` into `Foo.Foo.bar`.
Modelling qualification in the type system enables us to avoid the
problem by distinguishing between unqualified and qualified paths. A
function receiving an SymPath can safely qualify it, whereas a function
receiving a QualifiedPath should not further qualify the path. This
helps better express and ensure constraints across modules.
In addition, this commit also refactors a few functions where there was
opportunity to do so.
* refactor: remove eval call from `doc`
This can lead to problems where a doc call intended to be evaluated
later (in a macro body) is evaluated *immediately* resulting in a
binding being added to the wrong scope (see the function reverse in
core).
The reason this behavior crops up now is that a special case for
evaluating module contexts was removed last commit--this special case
caused problems of its own, and the real root of things stems from the
unnecessary eval call. Generally, evaling a doc call provides no benefit
other than making evaluation of the meta set immediate in the repl,
which is easy enough for one to do on one's own by calling eval where
needed.
* refactor: use do notation to clarify case qualification
* refactor: rename runQualified to unQualified
@eriksvedang pointed out the `run` prefix typically denotes a monad. As
`Qualified` is not monadic (no monad instance defined) we drop the `r`
to ensure we don't mislead readers.
* refactor: convert a few more binds to do notation
Do notation is generally clearer in cases where we use mapM, etc. We can
also leverage liftM frequently in the qualification functions to
transform a Qualified xobj back into an xobj for further use.
* refactor: temporarily restore special case in meta set
Meta set disallows setting the meta of a prefixed, absolute path such as
`Foo.bar`. It only allows relative, unqualified paths `bar` and uses the
current context to determine the appropriate module.
If we eventually throw and error from envInsertAt, we can remove this
special case. I intend to do that later, but for now we'll keep the
special case to make for a more pleasant user experience.
* fix: make set! work with dynamic args (thanks @hellerve)
Like `let` before it, we used to bind function arguments to their values
only, which wasn't accounted for in `set!` such that one could not
`set!` `i` in `defndynamic [i] ...`. To fix this for let bindings we
introduced a `LetDef` form for consistency with Def forms. This commit
renames `LetDef` to `LocalDef` and uses it as a value for function
arguments in addition to let bindings, ensuring `set!` works on function
arguments too. Big thanks to @hellerve for the suggestion!
* test: add regression test for set! on dynamic function args
* feat: add semigroup instance for Env and Context
Adds a semigroup instance for combining Envs and Contexts--this will be
necessary to ensure closure's are evaluated under the combination of the
context captured in the closure and the current global context during
evaluation.
The semigroup instances are left biased and will prefer bindings defined
in the left context/env argument in the case of conflicts (this is in
keeping with the implementation of `union` in Data.Map, the underlying
function powering this instance).
* fix: evaluate closures under the current context
Previously, closures were evaluated only under the context that was
stored during their creation. However, this can lead to issues where
closures do not resolve bindings to their latest definitions.
This commit leverages the semigroup instance of Context to evaluate
closures under the combination of the context captured during their
creation and the broader context during their evaluation/application,
preferring the context captured in the closure when bindings conflict.
This ensures that when we apply closures their local bindings still
resolve to definitions encapsulated in the closure, while other bindings
resolve to the definitions contained in the current overarching context
(instead of the old context captured by the closure).
* fix: fix bias for context env combinations in semigroup
Previously, the semigroup instance for Context was left-biased in all
the combinations of each context's environment. However, one usually
calls this function to combine some older context with a newer context,
intending to have the older context win *only* in the case of internal
environments.
This commit changes the behavior of the semigroup instance to better
reflect this use case. When one calls:
`c <> c'`
The envs in each context are combined as follows:
- internal: If conflicts occur, prefer the bindings of the context on
the LHS (the "older" context)
- global: If conflicts occur, prefer the bindings of the context on the
RHS ("newer" context)
- type: If conflicts occur, prefer the bindings of the context on the
RHS ("newer" context)
This ensures the resulting context uses the latest values in the chance
of conflicts in the global env/type env, and the older values in the
case of an internal env (a closure).
* test: add basic tests for closures
* refactor: rename test/closure -> test/dynamic-closure
Also updates the forms to test dynamic closures.
Notably, don't type check dynamic bindings (which can be set to
whatever) and eliminate a hang that resulted from not handling an error
at the end of the `set!` call. Also refactors some of the code in
efforts to make it a bit cleaner.
Also adds an error when `set!` can't find the variable one calls set!
on.
* feat: evaluate defs and defns in repl
* fix: better handling of static symbol evaluation
* refactor: include feedback by @scolsen (thanks)
* refactor: incorporate feedback by @eriksvedang into resolver code
* refactor: rename shouldResolve to resolver
* fix: don't set the inner env to globals in type mods
Previously, we set the inner environment of a type generated module to
the global env in cases where the overarching context didn't have an
inner env. This leads to problems where by the recognition of modules is
inconsistent, and one can't use the names of types as submodules in
certain circumstances.
This commit fixes that issue.
* refactor: refactor primitiveDefmodule
This refactor fixes a issues with meta information on submodules, for
instance, sigs on submodule functions used to result in a compiler error
about ambiguous identifiers. This fixes that.
Unfortunately, I don't have a precise idea about what exactly was wrong
with the original definition of this function. My suspicion is that the
recursion originally altered submodule paths in the wrong way, but I'm
not certain. In any case it's fixed.
* fix: ensure macros are expanded in the correct module
Previously, macro expansions folded over all forms after the top level
form, without performing any context updates on encountered
`defmodules`. This created an issue in which macro calls that produced
new bindings, "meta stubs", were *hoisted* out of submodules and into
the top-level module, creating duplicate definitions.
This commit fixes that issue by adding a special case for defmodule in
macroExpand.
* fix: ensure submodules and globals don't conflict
Previously, our module lookups during new module definition always
eventually fell back to the global environment, which caused submodules
that happen to share a name with a global module to be confused with the
global module. This change fixes that, so now one can define both
`Dynamic` (global) and `Foo.Dynamic` without issue.
* fix: remove old prefixes from vector tests
Commit 7b7cb5d1e replaced /= with a generic function. However, the
vector tests still called the specific Vector variants of this function,
which were removed when the generic was introduced. After recent
changes, these calls are now (correctly) identified as erroneous. My
guess is that they only worked in the past because of problems with our
lookups.
* chore: format code
* refactor: Move constraints test to own module, create TestLookup module
* refactor: Extracted 'Env' module
* refactor: Extracted 'TypePredicates' module
* test: First, very simple test
* refactor: Extracted 'Managed' module
* refactor: Add 'lookupBinder' function that doesn't return an Env (often what we want)
* refactor: Move out more stuff from Lookup
* refactor: Use new 'lookupBinder' in tons of places (avoids tuple)
* refactor: Got rid of monolithic 'recursiveLookupInternal'
* refactor: Avoid boolean blindness
* refactor: Better names for some lookup functions
* refactor: More logical order in Lookup.hs
* style: Use correct version of ormolu (0.1.4.1)
* refactor: Slightly more consistent naming
* refactor: Address @scolsen:s feedback
* Rename ty to xobjTy.
* Rename info to xobjInfo.
* Rename obj to xobjObj.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Address name shadowing.
* Remove some primes.
* compiler: refactor error reporting and add CARP_DIR check
Co-authored-by: Tim Dévé <TimDeve@users.noreply.github.com>
* set CARP_DIR in tests
Co-authored-by: Tim Dévé <TimDeve@users.noreply.github.com>
* Meta: Fix hangs on calls to meta functions
This commit fixes a subtle bug whereby setting the meta of an existing
binder would cause hangs. Ultimately, this still points to an underlying
issue in our Lookup code that causes such loops, but for now this at
least fixes the hang introduced by the doc call in `core/Tuples.carp`
(see issue #842).
The primary fix seems to be related to setting the parentEnv in a case
in which we failed to do so in `Eval.hs`. Additionally, our meta setting
macros call `eval` which causes them to be evaluated *immediately after*
expansion, causing them to be evaluated in the incorrect context in the
Tuples.carp case.
Additionally:
- Refactored primitiveMetaSet and primitiveMeta to make them cleaner.
- Only set `implements` early when we're certain we won't accidentally
overwrite the interface.
- Generalize DocStub to `MetaStub` so that it may be used for forward
meta declarations of any kind.
* Macros: Don't eval meta-set! macros
Calling eval in the body of the meta-set! macros can cause them to be
evaluated before anticipated, possibly setting the meta on a binding in
the incorrect environment. An exemplary case of this issue existed in
`Tuples.carp` (also fixed in this commit) whereby the generated
defmodule for a tuple type called doc, which was evaluated *before* the
emitted module, resulting in overwrites of global name docs instead of
the expected module function.
We retain `evals` in macros that are more useful in the repl, such as
`print-doc`. If a user wants to evaluated one of the meta-set macros in
the REPL, they'll need to add a call to eval.
* Macros: Restore calls to eval
Turns out the meta-set! macros *do* require calls to eval, unlike I
reported in the previous commit. This commit restores those and replaces
the `doc` call in `Tuples.carp` with a direct `meta-set!` to ensure we
have docs for those functions.
Also fixed a small error in implements primitive.
* Primitives: Refactor i->inner in primitiveImplements
