Closes#1741
Remaining blocker was to move logging to concrete robot.
Most notably, the `swarm-scenario` sublibrary has been purged of its temporal component, corroborated by the removal of the `time` package dependency.
Towards #1715 and #1043
Creates a new `swarm-scenario` sublibrary intended for scenario data that is independent of game state.
# Planned follow-ups
This PR is already pretty large, but there is still more that can be done regarding sublibrary reorganization/splitting.
* May want to pare-down a sublibrary exclusively for world-generation without all the other baggage of scenarios.
* `Swarm.Game.ScenarioInfo`, `Swarm.Game.Tick`, and `Swarm.Game.Scenario.Status` could probably be moved from `swarm-scenario` to `swarm-engine`.
Continuation of #1731.
Towards #1715 and #1043.
This refactoring step is a prerequisite for #1719 to extricate references to the `CESK` module from the base `RobotR` definition.
# In this PR:
* Extracts the `RobotContext` type to a new module
* Introduces a type family for the `RobotContext` field
Closes#1623.
Note that we still have to list both `vty-unix` and `vty-windows` in `stack.yaml` but that's OK; it just means they are both made available as extra dependencies, but only whichever is needed will actually be installed. In the codebase itself, we now get to avoid CPP on imports, and the code is quite a bit simpler.
Towards #1043
Extracts `Swarm.Game.*` modules to their own sublibrary.
There was already pretty good separation along this boundary; just had to move three functions into a new module `Swarm.Util.Content`.
Towards #1415.
Potential candidate for a web-based world display
Uses [pixi.js](https://pixijs.com/).
# Demo
1. Run swarm: `scripts/play.sh -i scenarios/Fun/horton.yaml --autoplay --speed 6`
1. Visit http://localhost:5357/play.html
Closes#1644.
The `"recognize"` property in scenario `.yaml` files is changed from a boolean to a list of "up" directions.
The structure recognizer adds a rotated copy of each supported orientation to its automaton. Rotational symmetry is accounted for to avoid duplicate work in the recognizer.
Also in this PR:
* Add cardinal directions to the JSON schema
* Tetromino packing challenge scenario
## Demos
scripts/run-tests.sh --test-arguments '--pattern "1644-rotated"'
### Structures dialog
### Tetromino packing
scripts/play.sh -i data/scenarios/Challenges/pack-tetrominoes.yaml --autoplay
Closes#1443.
Also added `-Wunused-packages` to clean up dependencies.
## Demo
This still works as usual:
stack run
Output editor keywords:
stack run swarm-docs -- editors --emacs
This is a continuation of #1652.
Most of the sub-records are bundled into `Swarm.Game.State.Substate`, but we create a `Swarm.Game.State.Robot` module just for robots.
We introduce a `zoomRobots` function so that applicable functions can operate directly on `Robots` state instead of `GameState`.
## Size comparison
### Before
| File | Lines |
| --- | --- |
| `State.hs` | 1569 |
### After
| File | Lines |
| --- | --- |
| `State.hs` | 812 |
| `Substate.hs` | 497 |
| `Robot.hs` | 395 |
| `Config.hs` | 21 |
## For follow-up PR:
- [ ] Remove exports of `_viewCenter` and `_focusedRobotID` from `Swarm.Game.State.Robot`
Both `execConst` and `stepCESK` are huge case statements. This refactor puts them in different modules.
Also many supporting functions for implementing commands in `execConst` are moved to their own modules.
Whereas `Step.hs` previously had `2931` lines, the new linecounts are:
| File | lines |
| --- | --- |
| `Step.hs` | 848 |
| `Arithmetic.hs` | 124 |
| `Const.hs` | 1687 |
| `Command.hs` | 417 |
The only tricky thing was that `execConst` is mutually recursive with `runCESK`. So to get them into different modules, I pass a wrapper of `runCESK` as an argument to `execConst`.
API docs are moved under `api/`, and the landing page for http://localhost:5357/ is now a static `index.html` page.
This paves the way for a JS-enabled web frontend demo.
Towards #1415.
## Uses
This capability could be used to quickly iterate on DSL world descriptions, e.g. when tuning noise parameters.
## Implementation notes
* For the hard-coded ANSI terminal color names, I chose RGB triples that matched my own terminal settings. This means that a rendered PNG might not exactly match one's own terminal colors.
* `Blank` terrain corresponds to a transparent pixel.
* Implemented parse-time validation of `attr` references. Previously, referencing a nonexistent `attr` by an entity would fail silently at runtime.
* Normalization: strings like `"rock"` now only exist once; the string is shared via toplevel variable definitions
* Entities and terrain have TUI-independent color definitions from which VTY Attrs are derived, but all TUI user-interface colors are defined only as VTY Attrs.
## Demos
Each pixel in the output image correponds to one world cell. To enlarge, can use [imagemagick](https://legacy.imagemagick.org/Usage/resize/#scale):
stack run -- map data/scenarios/classic.yaml --seed 0 --png -w 300 -h 200 && convert output.png -scale 800% out2.png
stack run -- map data/scenarios/Challenges/bridge-building.yaml --png && convert output.png -scale 800% out2.png
Closes#1569
## Performance
Path cache invalidation upon world modifications (i.e. entities inserted or removed) entails iterating over all of the previously-cached paths [here](https://github.com/swarm-game/swarm/pull/1595/files#r1390158411). For efficiency's sake, we avoid iterating over "all existing robots".
Any scenario that does not use the `path` command is entirely unaffected by this change.
## Demo
Previously, this demo was virtually unplayable, since when moving between widely-spaced "clusters" of flowers, an expensive A-star search was invoked at almost every tick. Now, the vast majority of moves utilize the cache, and the demo exhibits minimal stuttering (e.g. when a single A-star search is performed when moving between distant clusters).
scripts/play.sh -i scenarios/Fun/horton.yaml --autoplay --speed 7
### Event log
An event log specific to the path cache is maintained with its own ring buffer:
scripts/play.sh -i scenarios/Testing/1569-pathfinding-cache/1569-harvest-batch.yaml --autoplay
and view http://localhost:5357/paths/log
Added support for running on Windows, in command line or Powershell terminals.
Currently terminal emulators such as mintty, ConEmu, alacritty, etc are not supported.
Addresses issues #1607 and #53.
Closes#1575
Implements structure recognition.
## Features
* Structure browsing dialog (`F6`) that becomes available if a scenario declares any recognizable structures
* Automatically recognizes statically-placed structures upon scenario initialization, accounting for occlusion by other entity/structure placement
* New `structure` command for querying location of recognized structures (primarily intended for system robots and goal checking)
* Efficiently recognizes structures immediately upon completion
* Accounts for removal of structures
* Several new integration tests
* Structured web-interface log to help understand/debug the recognition process
* Re-uses much of the functionality built previously for defining structures (#1332)
Other changes:
* Improved validation for static structure placement (ensure valid structure names instead of failing silently)
* Moved a few functions (`getNeighborLocs`, `zoomWorld`, `entityAt`, `robotWithID`, `robotWithName`) out of `Step.Util` and into `State` so that recognizer initialization, which becomes a field in `GameState`, could use them
* split `scenarioToGameState` into pure and non-pure functions
## Optimizations
Scenarios that do not make use of structure recognition are entirely unaffected, performance-wise.
Some optimizations include:
* Structure definitions must "opt-in" to participate in automatic recognition
* Aho-Corasick automatons optimized by:
* only initiate structure search if a placed entity exists on a participating structure template
* initializing different automatons for each type of "placed entity"
* pruning inapplicable row candidates for 2-D search
The row-level structure recognition cache described in #1575 was not implemented; it's probably not worth the complexity cost.
# UI Demo
scripts/play.sh -i scenarios/Testing/1575-structure-recognizer/1575-browse-structures.yaml --autoplay
1. Press `F6` for Structure Browser dialog
2. View http://localhost:5357/recognize/log and http://localhost:5357/recognize/found
# Future improvements
* Refactor `State.hs` so that the new helper functions can live elsewhere
* Support non-rectangular recognizable structures
* Allow flip/rotate of recognizable structures
* Structure ownership by robots
* Consolidate code between the Goals and Structures modal dialogs, and the Achievements browser
* Enforce minimum/maximum dimensions for structure definitions
* add wave program and parametrise it to compare inlined/generic version
* use [`tasty-bench`](https://hackage.haskell.org/package/tasty-bench) library to show comparison
* move benchmarks to test folder as they can now share tasty code
* closes#1574
Using the recursive definition with ifs leads to a 3x slowdown:
```
wavesInlined
10: OK
361 ms ± 29 ms
20: OK
718 ms ± 35 ms
30: OK
1.066 s ± 28 ms
40: OK
1.437 s ± 37 ms
wavesWithDef
10: OK
1.052 s ± 51 ms, 2.92x
20: OK
2.117 s ± 34 ms, 2.95x
30: OK
3.144 s ± 80 ms, 2.95x
40: OK
4.191 s ± 91 ms, 2.92x
```
But if we just inline and simplify the code, we can remove the runtime overhead completely.
Closes#1436.
The schema `.json` files are now the authoritative source of truth for documentation.
Wrote a very simple parser for JsonSchema to extract the documentation from JSON.
Split the README.md into [static](c314cc50a1/data/scenarios/README.md) and [auto-generated](c314cc50a1/data/scenarios/doc-fragments/SCHEMA.md) parts.
Added a custom `"footers"` key to schema files to support inclusion of other markdown files for each object description.
# Schema doc regeneration
./scripts/regenerate-schema-docs.sh
Closes#353
Also adds a new top-level command to render a scenario map to the console.
Most of the work for this feature entailed identifying the subset of `GameState` that is actually needed for rendering the world, so that the required information can be retrieved from just the `Scenario` rather than having to instantiate an entire `GameState`.
# Potential follow-ups
- [ ] There is some noticeable lag when using the up/down arrow to navigate to any of the largely "procedurally generated" scenarios, e.g. `classic` or `creative`. May want to do caching of some kind. The other "challenge scenarios" render without perceptible delay.
- [ ] The heuristic for choosing the view center could be improved, possibly by defining new "hints" as part of the scenario schema.
- [ ] Rendering to the console could be augmented with color.
- [ ] Could render to other image formats like PNG or SVG
- [ ] account for a user-selected seed in the scenario launch parameters dialog
# Demos
## Scenario selection preview
## Command-line map rendering
```
~/github/swarm$ scripts/play.sh map data/scenarios/Challenges/ice-cream.yaml
OO
▒▒▒ ▒▒▒▒ OOOO
┌─┐▒┌──┐ MMMM
│B V6│ \ZZ/
└──────┘ \/
```
and
```
stack run -- map data/scenarios/Challenges/hackman.yaml
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
▒••••••••••▒••••••••••▒
▒o▒▒▒•▒▒▒▒•▒•▒▒▒▒•▒▒▒o▒
▒•▒▒▒•▒▒▒▒•▒•▒▒▒▒•▒▒▒•▒
▒•••••••••••••••••••••▒
▒•▒▒▒•▒•▒▒▒▒▒▒▒•▒•▒▒▒•▒
▒•••••▒••••▒••••▒•••••▒
▒▒▒▒▒•▒▒▒▒ ▒ ▒▒▒▒•▒▒▒▒▒
▒•▒ ▒•▒
▒▒▒▒▒•▒ ┌──=──┐ ▒•▒▒▒▒▒
• │ │ •
▒▒▒▒▒•▒ └─────┘ ▒•▒▒▒▒▒
▒•▒ ▒•▒
▒▒▒▒▒•▒ ▒▒▒▒▒▒▒ ▒•▒▒▒▒▒
▒••••••••••▒••••••••••▒
▒•▒▒▒•▒▒▒▒•▒•▒▒▒▒•▒▒▒•▒
▒o••▒•••••••••••••▒••o▒
▒▒▒•▒•▒•▒▒▒▒▒▒▒•▒•▒•▒▒▒
▒•••••▒••••▒••••▒•••••▒
▒•▒▒▒▒▒▒▒▒•▒•▒▒▒▒▒▒▒▒•▒
▒•••••••••••••••••••••▒
▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒
```
In preparation for #1483. `LogEntry` started life as something specific to robot logs. It then evolved to be used in the system log as well (see #1039 and #652), but in a sort of hacky way. This PR refactors `LogEntry` to be more generic.
- Move `Swarm.Game.Log` -> `Swarm.Log` since it's not specific to gameplay.
- Rename `ErrorLevel` to `Severity`, add a new `Info` level, and add a top-level `leSeverity` field
- Rename `leRobotName` to just `leName`, since it was already being used to name both robots and system components anyway
- Move robot-specific fields (*e.g.* robot ID) into the new `RobotLogSource` type, and add `LogSource` to differentiate between robot and system logs
- Various other minor improvements and tweaks
This adds two files. The first is the syntax file responsible for basic highlighting. The highlight categories are based roughly on those seen in the Emacs file ([swarm-mode.el](https://github.com/swarm-game/swarm/blob/main/editors/emacs/swarm-mode.el)). The second is the file for configuring the language server. Since it is based on Neovim's native LSP client, it only works with Neovim and not Vim.
`README.md` in the `editor` folder has also been updated to include instructions for setting these up.
- Add scrollbars on both the inventory and info panels
- Get rid of `. . .` at top and bottom of info panel, since we now have scrollbar as a visual indicator when there is more content
- Allow scrolling the REPL history (closes#60)
- PgUp/PgDown can be used to scroll (Shift+PgUp/Dn were not recognized on my system)
- Hitting any other key causes the view to jump back to the very bottom
- A computation finishing + printing an output also causes the view to jump to the bottom
- The REPL history is cached so that it only gets re-rendered whenever a new history entry (i.e. input or output) is added; this is needed since the history could get quite large.
- Also, fix the height of the key hint menus to 2 lines, even when the panel-specific menu (second line) is blank, so the world panel does not keep resizing as we move the focus between panels.
Thanks to @jtdaugherty for releasing `brick-1.10` with a new ability to specify blank space to the side of scrollbars; see https://github.com/jtdaugherty/brick/discussions/484 .
Also towards #1461 .
* add option `--width` to `format` CLI
* if width is not specified use terminal width
* if terminal width is unknown, use default (100)
This should help with testing (not so) long layouts like in #1473.
Blog post highlighting cool features in the newest release.
Also fix the version number in the .cabal file from 0.4 to 0.4.0.0. It's too late for the release (it always reports there is a new version available since 0.4.0.0 does not match 0.4) but oh well.
DSL for programming worlds, towards #1320 and #29 (and, indirectly, toward #50, since the world DSL should make a nice target for world saves) . Eventually this should be able to recreate all the world description/building features we have, though there is still a long way to go. But currently we can at least recreate the "classic" procedurally-generated world. I think this is a solid foundation we can merge as a first step, and then work on adding more features in subsequent PRs. Below are some notes that should help in reviewing. Note that the large number of files changed is due in large part to the elimination of the `default` field in scenario descriptions; see the "changed files" section below for an overview of the important/interesting changes.
Issues split off from this one: #1394#1395#1396#1397
Major changes
============
- New `data/worlds` subdirectory
- All `.world` files are parsed at load time and saved in a `WorldMap` which gets threaded through, similar to `EntityMap` (perhaps we should think about passing around a single record instead)
- Standard "classic" world
- Used to be `testWorld2`, defined in Haskell code; now it is defined via the DSL in `worlds/classic.world`. This should make it much easier to experiment with variations.
- We can now automatically extract entities mentioned in a world DSL term with `extractEntities`. There used to be an explicit list in `testWorld2Entities`, used to check pedagogy, generate documentation, etc., but it turns out it had (predictably) gotten out of date! This can't happen anymore.
- It is now referenced in several tutorials (backstory, farming, world101, speedruns, etc.)
- The `default` field of world descriptions is no more: one can use `dsl` to just specify a constant
- Note in `Swarm.Game.State`, `dslWF` and `arrayWF` are combined using the `Monoid` instance to create `wf`.
- `Erasable`
- It used to be the case that if some kind of default terrain + entity was specified (e.g. stone + water), any `map` would completely override the default. However, we want to move towards combining everything with a `Monoid` instance. But by default this means the default entity would show through anywhere the `map` did not specify an entity. So we need a way to explicitly "erase" an entity from a lower layer.
- If `e` is a `Semigroup`, then `Maybe e` is a `Monoid` where `Nothing` acts as an identity element. Likewise, `Erasable e` is a `Monoid` but adds two new elements: `ENothing` to be an identity, and `EErase` to be an *annihilator*. i.e. combining with `EErase` is like multiplying by zero.
- We can now specify `erase` as an entity to override entity underneath.
- There are several Haskell files with only changes related to `Erasable`, relating to e.g. the world editor, `PCells`, etc.; I'm not 100% sure I've always done the right thing here.
DSL overview
===========
- Integer, float, and Boolean literals. Note that `3` is *always* an `int`, and `3.0` is a `float`. It makes things much easier to not have to deal with `3` possibly being either `int` or `float`, though it does make things slightly more annoying for programmers.
- Standard boolean, arithmetic, and comparison operators
- `if ... then ... else ...`
- `<>` operator for combining via `Semigroup` instance
- Cell literals are enclosed in curly braces. Unlike the previous awkward world description syntax with one, two, or three-element lists denoting terrain, terrain + entity, or terrain + entity + robot, there can now be any number of elements in any order.
- `{foo}` will be resolved as either terrain, an entity, or a robot, whichever is successful. So if the names are unambiguous one can just write `{tree}` or `{stone}`.
- It is possible to explicitly indicate the type of cell value with syntax like `{entity: tree}` or `{terrain: stone}`.
- Multiple items separated by commas is syntax sugar for combining with `<>`. e.g. `{tree, entity: boulder, stone} = {tree} <> {entity: boulder} <> {stone}`.
- Ability to refer to the `seed`
- Refer to the current `x` or `y` coordinates or the `hash` of the current coordinates
- `let`-expressions for multiple variables: `let x1 = e1, x2 = e2, ... in ...`
- `overlay [e1, e2, ...]` layers `e1` on the bottom, `e2` on top of that, etc., using the `Semigroup` instance for world functions
- `"foo"` imports the DSL term in `worlds/foo.world`
- `perlin` function to generate perlin noise
- `mask` function to mask with a condition
Changed files
===========
- `Swarm.Util`: moved the `acquire` function here and gave it a more descriptive name.
- `Swarm.Doc.Gen`: can now extract mentioned entities directly.
- `Swarm.Game.Failure`: added new failure modes
- `Swarm.Game.Scenario.Topography.WorldDescription`: get rid of `defaultTerrain` field, add `worldProg` for DSL.
- `Swarm.Game.State`: see comment.
- `Swarm.Game.World`: a bit of reorganization. Added a bunch of modules under this.
- `Swarm.Game.World.Coords`: moved some code here from `Swarm.Game.World`.
- `Swarm.Game.World.Gen`: moved some things here from `Swarm.Game.WorldGen` (also deleted a bunch of irrelevant code), and also added the `extractEntities` function to get all entities mentioned by a DSL term.
- `Swarm.Game.World.Syntax`: raw, untyped syntax for world DSL terms.
- `Swarm.Game.World.Parse`: parser for world DSL terms. Fairly standard.
- `Swarm.Game.World.Typecheck`: takes raw, untyped terms produced by the parser and both typechecks and elaborates them into a simpler core language. An interesting feature is that the core language is *type-indexed*, so that the Haskell type system is actually ensuring that our typechecker is correct; every typechecked world DSL term value has a type which is indexed by a Haskell type corresponding to the type of the underlying DSL term. For example, `{entity: tree}` would have a type like `TTerm [] (World CellVall)` etc. Once terms make it through the typechecker, there cannot possibly be any bugs in the rest of the pipeline which would result in a crash, because the Haskell type system. (There could of course be *semantic* bugs.) Understanding exactly how the typechecker works is not too important. Of interest may be the `resolveCell` function, which determines how we decide what `Cell` is represented by a cell expression in curly braces.
- `Swarm.Game.World.Abstract`: compile elaborated, typechecked world DSL terms down into an extremely simple core language with only constants and function application. This gives us very fast evaluation of world DSL terms. Understanding this module is not really necessary but there is a link to a blog post for those who are interested in how it works.
- `Swarm.Game.World.Compile`: a further processing/compilation step after `Swarm.Game.World.Abstract`. Currently we don't actually use this, since it doesn't seem like it makes a big efficiency difference.
- `Swarm.Game.World.Interpret`: interpreter for abstracted world DSL terms.
- `Swarm.Game.World.Eval`: just puts together the pieces of the pipeline to evaluate a typechecked world DSL term.
- `Swarm.Game.World.Load`: just loading world DSL terms from disk.
Closes#1122 . General principles:
- Use `SystemFailure` as error rather than `Text` as much as possible, and use `prettyFailure` only at the very top level.
- Replace `ExceptT` with `Has (Throw SystemFailure)` constraint.
- Use `Accum (Seq SystemFailure)` constraints to accumulate warnings that should not abort computation, rather than returning a pair of a list of warnings + result.
- Use `Has (Lift IO)` constraint instead of `MonadIO`, which means using `sendIO` instead of `liftIO`.
- In general, use `runThrow` to dispatch a `Throw` constraint (results in returning an `Either`, just like `runExceptT`), and `runM` to dispatch a final `Lift IO` constraint to result in an `IO` computation.
- Use `withThrow` to adapt from one type of error to another.
