Towards #1043.
The eventual goal of this sublibrary split is to have a self contained library that can compose 2D grids of arbitrary content (perhaps colored pixels, or boolean values). This could be useful outside of the `swarm` game.
I would also like to write unit tests for the structure recognizer that are independent of the `Entity` type.
# Major Changes
## Direction module
* Moved `Swarm.Language.Syntax.Direction` to `swarm-util`, since both `swarm-lang` and `swarm-topology` depend on it, but not on each other.
* Removed the re-export of direction things from `Swarm.Language.Syntax`
## Structure module
The `Swarm.Game.Scenario.Topography.Structure` module has been split into two:
* `Swarm.Game.Scenario.Topography.Structure`
* `Swarm.Game.Scenario.Topography.Structure.Type`
The former retains the YAML parsing logic. The latter is agnostic of `Enitiy` type and the palette .
At some future point, I might want to move the YAML parsing to this sublibrary while still retaining independence of `Entity` type.
## Structure recognizer
The structure recognizer is independent of the content of Cells (i.e. it does not need to know what an `Entity` is), except:
1. during initialization
2. when retrieving the original cell content after recognition
Type parameters for three kinds of data have been added to the recognizer:
1. `Cell`/`PCell`
2. `Entity`
3. `EntityName`
Eventually it may be possible to eliminate one or two of these type parameters, with some refactoring.
Adds type synonym declarations to the language, like so:
```
tydef Maybe a = Unit + a end
def lookdown : Cmd (Maybe Text) = scan down end
> lookdown
it2 : Maybe Text = inl ()
```
`tydef` behaves very similarly to `def` except that it defines a new (parameterized) type instead of a new term.
Note that higher-kinded types are not yet supported; for example we cannot say `tydef Foo f = Unit + f Int` (in fact this would be a syntax error since type variables cannot be at the head of an application). However there is nothing stopping us from adding that in the future; I just wanted to keep things simple for now.
If you know of any scenario code that would be a good candidate for using type synonyms, let me know --- it would be nice to update a few scenarios as a better stress test of type synonyms before merging this.
Closes#153.
Before:
```
ghci> let s = "[{ \"radius\": 2, \"density\": \"x\"}]"
ghci> eitherDecodeStrict s :: Either String GrowthSpread
Left "Error in $: parsing Growth failed, expected Object, but encountered Array
```
The error msg above is misleading because we weren't trying to parse `Growth` datatype but `GrowthSpread` instead.
After:
```
ghci> let s = "[{ \"radius\": 2, \"density\": \"x\"}]"
ghci> eitherDecodeStrict s :: Either String GrowthSpread
Left "Error in $: parsing GrowthSpread failed, expected Object, but encountered Array"
```
Taking a stab at refactoring `FromJSON` instances. Initially while skimming through the swarm code, I found multiple occurrences of using applicatives for `FromJSON` instances but was hesitant to bring it up. In my experience, it's almost always ergonomic to use `do` notation with `RecordWildCards`.
Closes#1878
Fixes#1625.
Before: `path : Unit + Int -> Int * Int + Text -> Cmd (Unit + Dir * Int)`
After: `path : (Unit + Int) -> ((Int * Int) + Text) -> Cmd (Unit + (Dir * Int))`
We essentially consider sum, product, and arrow to all have the same precedence level; but arrows still associate to the right.
Closes#1291. The idea is that log entries produced by `log` or `say` will always be shown; but other log messages (error messages, status reports, etc.) are "ephemeral", meaning they will be shown as long as they are the most recent log entry, but will no longer be shown once a new log entry is added. The motivation is that it is often unhelpful to see one's log full of error messages.
- All the log messages are still retained in the `RobotLog`; we just filter out some entries when displaying the log in the info panel.
- All unfiltered log entries are still shown in the Messages dialog.
- Also added a `run: OK.` log entry for successful execution of `run`, so that it in the common situation when you are trying repeatedly to `run` a file and then fixing type errors, you can immediately tell when you are successful since the error will disappear to be replaced by an `OK` message.
To test this, start a new game and try executing some commands that cause an error (e.g. `run` a non-existent file, or execute `move` with the base robot, etc.) as well as executing `log` or `say`. Observe that `log` and `say` entries persist, whereas error messages are shown only until a new log entry is added.
This PR adds:
- A new entity called `lens`
- Recipe: 1 `glass` + 1 `mithril` with a required `drill` (idea: you grind the piece of glass using a mithril-coated drill bit)
- For now it does not convey any capabilities, but it seems ripe for some sort of pun on lenses.
- A recipe for `detonator`:
- 1 `string` + 10 `curry` + 1 `lens`
- Idea: curry is explosive, so you attach a string as a fuse and then ignite it by focusing sunlight with the lens.
Because of the `mithril`, these are not so easy to craft (Level 13 and 14, respectively) but I don't know any particular reason that these need to be easy to craft. If we wanted, we could easily remove the `mithril`, which would result in them being Level 6 and 7 instead.
Closes#1017.
We used to represent types with a different constructor for each different sort of type, e.g. there was a constructor `TyFunF` which took two types as arguments, etc. This refactoring creates a new type `TyCon` which has things like `TCFun` and `TCBase` and represents various types as an application of a `TyCon` to some arguments. For example function type `t1 -> t2` would be represented no longer as `TyFunF t1 t2` but instead as `TyConApp TCFun [t1, t2]`.
This is slightly more roundabout, and it does make parsing slightly trickier, but it greatly simplifies and shortens code for e.g. unification, extracting free variables, etc. (because what used to be many essentially identical cases now turn into a single case). It also paves the way for #1865, so we have a way to represent new type constructors defined by the user. E.g. if the user defined `tydef Maybe a = Unit + a` then we would represent the type `Maybe Int` as something like `TyConApp (TCUser "Maybe") [TyBase BInt]`.
This also means that types like `Int Unit` are no longer a parse error, so we need kind checking to rule out applications of type constructors to the wrong number of arguments; this PR adds such kind checking as well. Having some sort of kind checking is unavoidable when allowing the user to define their own new type constructors (with an arbitrary number of arguments).
There were a bunch of fused-effects related utils that lived in a module in `swarm-scenario`; but all of them were completely generic except for the `simpleErrorHandle` function which is specific to `SystemFailure`. So, this PR:
- Moves `simpleErrorHandle` to `Swarm.Game.Failure`
- Moves the rest of the module into the `swarm-util` package so it can be used elsewhere.
This is another refactoring in preparation for #1865.
Splitting out an independent refactoring to simplify #1865. There were many places in the code where we were pattern-matching on `ProcessedTerm` and/or `Module`, sometimes only to extract a single field, *e.g.*
```
ProcessedTerm (Module s _) _ _ <- maybeInitialCode
```
This was already getting a bit out of hand, and it also meant that any time we wanted to add an extra field to either `ProcessedTerm` or `Module` (as I will be doing in #1865), we would have to change every single such place in the code to add an extra wildcard pattern.
This PR is a refactoring to
1. Derive lenses for all the fields of `ProcessedTerm` and `Module`, and
2. Prefer extracting named fields via lenses over pattern-matching in most places in the code.
There were a few places in the code where we destructure a `ProcessedTerm` and use almost all the fields (e.g. when initializing a robot machine); I left those alone, first because extracting all the individual fields would be tedious, but more importantly because if we ever added any new fields in the future, it's likely we would actually want to use them in those contexts as well.
Closes#1533
# Demo
```
scripts/test/run-tests.sh --test-options '--pattern "sow"'
```
or
```
scripts/play.sh -i data/scenarios/Testing/1533-sow-command.yaml --autoplay
```
## Other changes
* Introduced the `ChildInheritance` type to specify how the `Display` attribute is inherited by built child robots.
* Introduce the `mature` property of `growth`. For example, it doesn't make much sense to plant an `acorn` and call the resulting, fully-grown plant also an `acorn`. Instead, an `acorn` matures into an `oak`.
Closes#1780.
This change allows "child" structures to be placed outside of the bounds of their "parent" map. Until now, any child structures that exceed the bounds of their parent were simply truncated.
# Demo
```
scripts/play.sh -i data/scenarios/Testing/1780-structure-merge-expansion/structure-composition.yaml
```
Closes#1847.
## Demo
### Production
https://swarmgame.net/list-games.html
### Local testing
```
tournament/scripts/demo/server-native.sh
```
and
```
scripts/test/run-tests.sh swarm:test:tournament-host
```
## Authentication flow
1. Users are represented by a GitHub username (primary key) and an "authentication cookie" in the SQLite database.
2. Site prompts user to login when the client's cookie is nonexistent or does not match any user in the database.
3. GitHub flow:
1. Clicking the "Login" link redirects user to the GitHub login page.
2. GitHub sends a `code` to our callback URL.
3. use that `code` to get an "access token"
4. use the "access token" to look up the username of the person who is logging in.
5. generate and store a new cookie in the database row for that username
6. set the cookie value on the user's client.
4. As long as the client keeps sending the cookie value known to the server, all uploads/activity will be attributed to their GitHub username.
## New features
* Login/Logout
* All uploaded content is attributed to an authenticated GitHub user
* Separate pages for scenario lists and solution lists
* Download a solution file
This is a followup on top of #1583 which turns `swarm format` into a tool for porting existing Swarm code into the newest syntax, via an extra `--v0.5` argument. In particular, this PR:
- Generalizes the parser to take a configuration record, which among other things contains the language version being parsed.
- Adds code to allow the parser to parse either the current syntax or one version ago (when types did not start with capital letter) depending on the version in the configuration.
- The idea is to have the parser always support the current version and one older version, so we can always upgrade version n to version n+1.
- Adds a new flag `--v0.5` to the `format` subcommand which causes the input to be parsed in v0.5 mode. However, the output of `format` will always use the latest syntax. Thus, `swarm format --v0.5` reads code in v0.5 format and prints it in the latest format, so this can be used to automatically port existing `.sw` files.
This PR also makes a few minor improvements to pretty-printing.
Closes#1547 , which was caused by misspelled/nonexistent types being interpreted as type variables. In #1550 I proposed one solution to the problem, namely, to stop implicitly quantifying types and require explicit `forall` on any polymorphic type. This PR represents an alternative solution: to keep implicit quantification but require all types to start with an uppercase letter, as in Haskell. I think I like this one better but I'm open to feedback.
Specifically, with this PR:
- All built-in type constructors (`Int`, `Cmd`, `Unit`, etc.) must start with a capital letter
- Type variables:
- Must start with a lowercase letter or underscore
- May not be named a lowercase version of a type constructor, *e.g.* `int`
- This is important so that old code with lowercase types is not silently accepted by parsing the former types as implicitly quantified type variables; even in cases where old code no longer typechecks, this will give better error messages.
- Term variables:
- May start with upper- or lowercase
- May be named lowercase versions of type names, *e.g.* `let f : Int -> Int = \int. int + 1` is fine
This PR obviously represents a bigger breaking change. Once we merge this we might consider adding an option to `swarm format` to be able to parse old code with lowercase types and then reformat it with uppercase, to aid in porting code.
Once this is merged I will also regenerate the wiki pages that mention types.
Closes#1550.
Hello,
Splitting `Swarm.Language.Syntax` into submodules reduces the compilation time quite a bit. Currently, it takes the most of time out of all the modules. Here's a chart:
That's ~20s for simplifier (with overall compilation time of 4.3 mins)
Just taking out the types and functions related to `Constants` and putting them in a separate module results in:
Thats ~8.5 seconds for simplifier (with overall compilation time of 3.9 mins)
Just for added info, timings taken on a 2019 macbook with i7 processor (6 core and 32 GB RAM)
Changes include:
- `Swarm.Language.Syntax` split into the following modules:
- `Swarm.Language.Syntax.Comments` - Types for working with comments.
- `Swarm.Language.Syntax.Type` - Core types (`Syntax'`, `Term'` and `DelayType`)
- `Swarm.Language.Syntax.Pattern` - Pattern synonyms for untyped terms
- `Swarm.Language.Syntax.Loc` - Types for working with location in source code (`SrcLoc` and related types)
- `Swarm.Language.Syntax.Util` - Helper functions (Eg: `mkTuple`, `freeVarsS` etc)
Compilation chart after the split:
`Swarm.Language.Syntax` is no longer the most time taking module.
Closes#1844
This PR does three main things:
1. Insert parsed comments into the AST
2. Extend pretty-printing to include comments
3. Extend the `format` subcommand with a few additional options and move the code to `Swarm.Language.Format`.
The pretty-printed code-with-comments is OK but not great. It does a reasonable job with comments in standard-ish places; for example, it turns
```
// This function increments a number
def incr : int -> int =
\n. n + 1
end
/* This command does some stuff. It is super
cool and important. */
def stuff : cmd unit =
move;
move;
move; // the third move is important
move;
end
```
into
```
// This function increments a number
def incr: int -> int = \n. n + 1 end;
/* This command does some stuff. It is super
cool and important. */
def stuff: cmd unit =
move;
move;
move // the third move is important
;
move
end
```
which is good other than the fact that it moves the inline comment after `move;` to before the semicolon.
However, it turns this:
```
// This function does a cool math thing
def foo : int -> int = // This is an optional type signature
// pre
\n. n + 1 // add one
end
/* This is a
block comment which
spans multiple lines */
def bar : int -> int // Another type signature, = on the next line
= \x. foo /* very important to use foo here */ (foo x) // very cool implementation
end
def baz : cmd unit =
move;
move;
turn left; // don't forget to turn left!
move
end
// And one last thing
```
into this:
```
// This function does a cool math thing
def foo: int -> int
= \n.
n + 1 // add one
end // This is an optional type signature
;
/* This is a
block comment which
spans multiple lines */
def bar: int -> int
= \x.
foo /* very important to use foo here */ (
foo x // very cool implementation
)
end // Another type signature, = on the next line
```
which has several obvious problems. I think I know what the problem is in most cases; it will just require more engineering and special cases to get the output to look nicer, but I honestly don't really want to spend more time on this right now. I'm hoping we can merge this as is (since it is still better than the status quo, namely, deleting all comments) and continue to improve it in the future.
The important point is that I ran the code formatter on every single `.sw` file in the repository and then re-ran the test suite; all the tests passed. So at least `swarm format` does not seem to break anything even if the output does not look the best.
Closes#1467 .
As a follow-up to #1834:
- Update the sublibrary graph generation script so it works with the no-longer-linear graph
- Regenerate the actual image
- Update the `.cabal` description to match
This is a rework of #1798 to facilitate a simpler web stack.
# Demo
View http://swarmgame.net/
NOTE: Requires IPv6
# Motivation
Hosting cost is a main motivation. Cost per month for an EC2 instance, RDS, and the requisite other services approaches >$50 per month. In contrast, the lowest-tier Lightsail instance is $3.50/month.
The deployment process is of course simplified.
An incidental benefit to using SQLite is reduced latency of web requests; we no longer need to fetch credentials from an AWS API to connect to Postgres.
## Changes
Major changes:
* Use `sqlite` instead of `postgres`
* Use Docker to build a statically-linked deployable binary, rather than deploying the app within a Docker image
Fortunately, the API of `sqlite-simple` is near-identical to that of `postgresql-simple`, so most of the code change there is just to rip out AWS-specific stuff and Postgres connection info. I have no hesitation to delete this code since if we ever want to use the previous stack again, we can just look at #1798.
I noticed the parser was getting kind of unwieldy, and I'm going to be adding even more code to it as part of #1467, so I decided to take the opportunity to refactor first.
This PR introduces no behavior changes, it is purely refactoring.
Towards #1467. This is incomplete but I thought it would be useful to split up the work into multiple PRs to avoid having a really massive one at the end.
This PR accomplishes a few things:
- Creates a new data type `Comment` to record the text of a comment, along with a bit of metadata giving its source location, whether it is a line or block comment, and whether it was on a line by itself or at the end of a line with some other non-comment tokens (this information will be used to decide which AST node to associate the comment with).
- Adds a field to store comments in every `Syntax'` node. Note this is currently unused, since we haven't yet implemented the logic to insert comments into appropriate AST nodes (that will come in a later PR).
- Note it's a bit annoying that the number of fields of each `Syntax'` node is growing. I did look into consolidating and generalizing `Syntax'` to just have a single field for arbitrary annotations, but the changes required seemed annoying enough that I didn't want to bother.
#1834 created a `swarm-tui` sublibrary for the remaining source modules that weren't in a sublibrary of their own, but I forgot to actually move them all into a `swarm-tui` folder.
It is more intuitive in scenario design for "later" things to be placed "on top of" earlier things.
There were only a small number of scenarios for which placement order was relevant, so I fixed them.
Make sure Swarm builds with the most recent versions of dependencies. The only one that is still lagging is `filepath`, but `servant` doesn't support `filepath-1.5` so Swarm can't either.
The only real update needed was due to some breaking changes in `aeson-2.2`, but the necessary changes were very minor. Also fixed a few places where GHC 9.8 is generating additional warnings.
Also update our CI: `haskell-ci` recently dropped support for `hlint`, in favor of projects defining their own standalone `hlint` workflows.
This does two main things to simplify the review of #1826:
* Add the `truncate` option to structure placement. By default it shall be `true` to maintain the status quo behavior
* Extract a `validatePlacement` function to improve readability
Recently the test suite started failing for me with an error about not finding an `00-ORDER.txt` file in `doc-fragments`. I'm not sure why it never failed before, but the reason for the failure is clear enough: any folder under `data/scenarios` which does not start with an underscore is assumed to contain scenarios.
I noticed this when regenerating the recipe graph on the wiki and saw that `tea leaves` was sitting there with no input. `tea leaves` are obtained by harvesting `tea plant`s (which now occur naturally in the world) but we were not taking such "yield" pairs into account when creating the graph. This PR adds a directed edge for each entity that `yields` another.
The image is already updated on the wiki with the output from this PR:
https://github.com/swarm-game/swarm/wiki/Recipes-cheat-sheet
Split the `Structure.hs` module in preparation for #1780. This refactoring entails no functional change.
# Also included
* follow-up to #1804 to remove REPL re-exports
* reformat `swarm.cabal` for the latest version of `cabal-gild`
The only reason the `swarm-integration` test had `swarm-tournament` as a dependency was because of a few validation functions in `Swarm.Web.Tournament.Validate`. However, this seemed to me like a strange place for those validation functions to live, since they had nothing to do with running a tournament in particular, and it meant that running the integration tests required having postgresql installed even though no databases are used in the integration tests at all.
This PR moves those validation functions to two more generic places: the first function to extract text from a sequence of log messages now lives in `Swarm.Log`, and the other two functions to check for bad errors and run a `GameState` to completion now live in a new module `Swarm.Game.Step.Validate`.
Also, fix a bug---it didn't work before if an appearance string of length 5 was given.
Closes#1230.
I don't know how to write an automated test for this, but you can see it working by *e.g.*:
- Start a creative game
- Execute e.g. `appear "DNESW" (inr "rock")`
- Observe that the base now looks like a rock-colored `N`
- Now do some `turn` commands (including `turn down`) and observe the appearance changing to e.g. `S` when facing south
A few minor updates:
- Fix formatting of code snippet in `Elmer's glue` entity description
- Fix type in conditional tutorial that referred to `robot` instead of `actor`
- Help now describes ctrl-q as "quit or restart"
- Update crash tutorial log message with hint about the static we will see if we salvage the currently viewed robot
Closes#1661; towards #154.
`unification-fd` is very powerful, and extremely fast, but it was written a long time ago and its age shows. It was not possible to incorporate it into our effects system in a nice way, necessitating the use of concrete monad transformers in the typechecking code. In addition it is impossible to customize, and we have been contemplating new type system features such as #153 and #154 that turn out to require hooking into the way the unification algorithm works (see https://github.com/swarm-game/swarm/issues/154#issuecomment-1822877487 for more details).
This PR thus removes the dependency on `unification-fd` and implements our own version of unification. It is not quite as fast as `unification-fd` but I consider the slowdown acceptable in order to gain e.g. recursive types. And of course there is also room to optimize it.
The custom `UTerm` from `unification-fd` is replaced with the standard `Free` (free monad) construction from the `free` package, and the custom `Fix` from `unification-fd` is replaced with the one from `data-fix`.
We also get rid of the `unifyCheck` function, which used to be a quick short-circuiting way to check whether two types definitely did not unify or might unify, allowing us to give better error messages more quickly. Now, the `=:=` unification operator itself just does this.
Towards #1797
Hosts an online repository of scenarios, against which solutions may be submitted. This is the foundational layer that may support more structured "tournaments", scenario ranking, or other social activity.
# Demo
## Live server
http://swarmgame.net/list-games.html
One can use the [`submit.sh`](https://github.com/swarm-game/swarm/pull/1798/files#diff-450877e3442a0ec1c5cbe964808a263d67f1e680d3aa3c3bf9ae6f51eca682fb) script and see valid uploads reflected live on the website.
## Local testing
### Automated tests
These are database-agnostic.
scripts/run-tests.sh swarm:test:tournament-host
### Manual tests
These test database interactions. It requires first setting up a local Postgres server.
1. Start `tournament/scripts/demo/server-native.sh` in one console
2. Run `tournament/scripts/demo/client/test-cases/local/good-submit.sh` in another
# Features
* Upload and validates scenarios
* Download scenarios with solution redacted
* Submit, validate, execute, and score solutions
# Key components
* Servant server
* Hosted on AWS in a Docker container
* Stores to a Postgres database in Amazon RDS
* Shares some code with the integration tests for evaluating scenarios and solutions
The production database uses IAM to manage logins. The web app uses the AWS API to fetch a "token" which can be used to log in instead of a password. This avoids having to store a password on the server.
# TODO
- [ ] User authentication (GitHub OpenID?)
Closes#1684Closes#1262
# Demo
A simple "puzzle" that makes use of consumables:
scripts/play.sh -i data/scenarios/Testing/1777-capability-cost.yaml --autoplay
Demo of enabled commands and costs display in left pane:
scripts/play.sh -i data/scenarios/Testing/1262-display-device-commands.yaml
# In this PR
* Supports specifying capabilities both as plain lists and as maps from capabilities to ingredients in YAML
* JSON Schema support for both capability specification styles
* New integration test
* Removed redundant `tshow` implementation from `Swarm.Doc.Util`
# Entity lookup approaches
The cost of exercising a capability in terms of "ingredients" is specified by the `_entityCapabilities` field within an `Entity` definition. Each ingredient itself is an entity, specified by name.
For many purposes, having ingredients just of type `EntityName` is sufficient. But for `Swarm.Game.Recipe.findLacking` in particular, the ingredients list must be actual `Entity` objects, not just names of entities. So at some point the names need to be looked up in the global entity map to be promoted to `Entity` objects.
The full list of entities is not available at `Entity` parse time to look up an ingredient entity by name, so we cannot store ingredients lists of type `(Count, Entity)` within a parent `Entity` object.
Approaches considered were:
* Store a copy of the `entityMap` in `RobotR`, for use by the `equippedDevices` lens
* Introduce a type parameter to `Entity` representing a "parse phase"
* **Allow a redundant "entity lookup" (and validation) at command execution time**
## Store `entityMap` in `RobotR`
One approach explored was to add a field to `RobotR`:
```
, _globalEntityMap :: EntityMap
```
This allowed the `equippedDevices` lens implementation to promote the `EntityName`s to `Entity`s when setting the value of the `_robotCapabilities` field. However, it was rather invasive as it entailed threading the `EntityMap` through many new code paths.
## `Entity` type parameter
Currently, `Entity` has a field:
```
_entityCapabilities :: SingleEntityCapabilities EntityName
```
This would entail a huge refactoring, with:
```
data Entity e = Entity
...
, _entityCapabilities :: SingleEntityCapabilities e
```
At initial parse time we would obtain a list of `Entity EntityName`, but somewhere later during `Scenario` parse time, we can do another pass to obtain `Entity Entity` objects.
This would at least have the advantage of doing the entity lookup/validation on ingredient lists in exactly one place, at parse time.
## Defer `EntityName -> Entity` promotion to command execution time
This is what is implemented in this PR. The global set of capability costs is validated at scenario parse time. But it is also redundantly validated in the `payExerciseCost` function, which is not ideal.
This is a random refactoring that does two things:
* Improves speed of `scripts/gen/render-sublibrary-dependencies.sh` script
* Refine imports of the `TestModel.hs` (renamed to `TestRepl.hs`) unit test
It appears that the only reason that the `swarm-unit` test suite imports the entire `swarm` sublibrary (hence the `brick` package), is due to `TestRepl`, which (now) imports `Swarm.TUI.Model.Repl`. Perhaps it would make sense to separate these at some point?
Prerequisite to #1798
## Changes
* Pass the final `TickNumber` count as a member of `Won` constructor so that it can be used in scoring submitted solutions
* Extract a helper function `codeMetricsFromSyntax` that can be reused by tournament server
* `ToJSON` instance for `ScenarioMetadata`
* New script `list-sublibraries.sh` to list the sublibraries defined in a package
