Closes#1641
The `data/terrain.yaml` file is now the authoritative source of terrains, though `BlankT` is still a hard-coded special case.
I have not changed the underlying integer representation of terrain in the world function, which means that the `terrainIndexByName` Map in the `TerrainMap` record is needed for translating between `Int` and `TerrainType`.
# Demo
scripts/play.sh -i data/scenarios/Testing/1775-custom-terrain.yaml
# Changes
* There used to be a function called `integrateScenarioEntities` that combined the `EntityMap` stored in the `Scenario` record with the global entity map. However, the global entity map is accessible at parse time of the `Scenario`, so we do the combining there and only ever store the combined map in the `Scenario` record.
* JSON Schema for terrain
* Removed the distinction between "World" attributes and "Terrain" attributes
* Unit tests for scenario-defined terrain and related validations
* Validate existence of referenced terrain at scenario parse time
* Validate attributes referenced by terrains at parse time
scripts/play.sh -i data/scenarios/Challenges/gallery.yaml --autoplay
## Code changes
In addition to the new scenario, this PR includes a code change to allow the order of items returned by the `tagmembers` to be specified by the scenario author.
Measures the volume of an enclosed space.
A useful alternative to the `path` command for goal checking.
## Demo
scripts/play.sh -i data/scenarios/Testing/1747-volume-command.yaml --autoplay --speed 2
* `tea plant` (which now spawns rarely in a certain biome of the classic world) can be harvested to yield...
* `tea leaves`, which can be combined with `water` to produce...
* `cup of tea`, which, along with `atomic vector plotter` (= `rubber band` + `typewriter` + `compass` + `counter`), are ingredients for...
* `infinite improbability drive`, which enables `teleport`.
When used by a privileged robot (system robot or in creative mode), the `teleport` command continues to function as before. When used by an unprivileged robot, that is, via the `infinite improbability drive` device, a random entity spawns somewhere near the target location.
Closes#1041.
Closes#845
**All changes are non-significant whitespace.** This can be verified with:
git show --ignore-all-space --ignore-blank-lines
in which the remaining changes are only elimination of manual word-wrap in descriptions.
Normalization is accomplished with this command:
scripts/normalize-all-scenarios.sh
This is an initial normalization pass that shall be a pre-requisite for #1713.
Towards #845
This is to avoid the undesirable transformation by `yq` of whitespace-preserving string fields into quoted one-line strings, described in https://github.com/swarm-game/swarm/pull/1706#discussion_r1442274007.
Command:
find data/scenarios -type f -name '*.yaml' -print0 | xargs -0 --max-args 1 sed -i -e 's/[[:blank:]]\+$//'
Closes#1644.
The `"recognize"` property in scenario `.yaml` files is changed from a boolean to a list of "up" directions.
The structure recognizer adds a rotated copy of each supported orientation to its automaton. Rotational symmetry is accounted for to avoid duplicate work in the recognizer.
Also in this PR:
* Add cardinal directions to the JSON schema
* Tetromino packing challenge scenario
## Demos
scripts/run-tests.sh --test-arguments '--pattern "1644-rotated"'
### Structures dialog
### Tetromino packing
scripts/play.sh -i data/scenarios/Challenges/pack-tetrominoes.yaml --autoplay
Previously, the logic to wrap horizontally was hand-coded, both for the player (using a dedicated monitor robot) and for ghosts (who, as system robots, implemented the teleportation themselves).
Now that we can use built-in portal functionality (#1356), no custom logic is needed to handle the teleportation. A (much-simplified) monitor robot still observes the base robot to award the hidden "World wrap" goal.
## Demo
scripts/play.sh -i data/scenarios/Challenges/hackman.yaml --autoplay --speed 6
Towards #1415.
## Uses
This capability could be used to quickly iterate on DSL world descriptions, e.g. when tuning noise parameters.
## Implementation notes
* For the hard-coded ANSI terminal color names, I chose RGB triples that matched my own terminal settings. This means that a rendered PNG might not exactly match one's own terminal colors.
* `Blank` terrain corresponds to a transparent pixel.
* Implemented parse-time validation of `attr` references. Previously, referencing a nonexistent `attr` by an entity would fail silently at runtime.
* Normalization: strings like `"rock"` now only exist once; the string is shared via toplevel variable definitions
* Entities and terrain have TUI-independent color definitions from which VTY Attrs are derived, but all TUI user-interface colors are defined only as VTY Attrs.
## Demos
Each pixel in the output image correponds to one world cell. To enlarge, can use [imagemagick](https://legacy.imagemagick.org/Usage/resize/#scale):
stack run -- map data/scenarios/classic.yaml --seed 0 --png -w 300 -h 200 && convert output.png -scale 800% out2.png
stack run -- map data/scenarios/Challenges/bridge-building.yaml --png && convert output.png -scale 800% out2.png
Structure recognition already supports overlapping bounding boxes between two structures (where one of the structure templates has vacant cells). This PR adds test coverage for this.
More importantly, this PR makes sure that structures can be recognized even with non-participating entities within their bounding box. The same integration test (`1575-bounding-box-overlap.yaml`) covers this case as well.
A "non-participating entity" is an entity that is not an ingredient to any structures participating in structure recognition. Such entities are "masked out" when the world cells (i.e. the "haystack") are queried to apply the Aho-Corasick matcher to.
## Example
Given a structure defined as `boulder`s (`@`) in a backwards "L" shape, it will be recognized despite a `mountain` (`A`) within its bounding box:
This is because `mountain` is not a member of any recognizable structures.
## Caveats
This PR strictly increases the situations in which valid structures may be recognized.
However, it is still the case that encroaching entities that **are** members of some structure template will thwart structure recognition. One consequence of this is that the order in which structures are completed can matter.
If some partially-built but incomplete structure lies within the bounding box of another candidate structure, the candidate structure will not be recognized as "complete" unless (1) the offending entities are removed first, or (2) the other structure is completed first.
Closes#1631
## Design
* Entities have a new property: a `Set` of textual tags.
* Two new commands are introduced:
* `HasTag` checks whether a single entity has a given tag
* `TagMembers` allows cycling through all members with a given tag
* `TagMembers` may be considered more powerful than `HasTag`, so has its own separate capability (`CTagmembers`).
* A map is computed at scenario initialization to facilitate `TagMembers` lookups.
* Tag names are highlighted in yellow in markdown.
## Demo
scripts/play.sh -i scenarios/Testing/1631-tags.yaml --autoplay
## Other changes
* Incidentally, changed `knownEntities` from a list to a `Set` so that `Set.member` can be used instead of `elem`.
Closes#1569
## Performance
Path cache invalidation upon world modifications (i.e. entities inserted or removed) entails iterating over all of the previously-cached paths [here](https://github.com/swarm-game/swarm/pull/1595/files#r1390158411). For efficiency's sake, we avoid iterating over "all existing robots".
Any scenario that does not use the `path` command is entirely unaffected by this change.
## Demo
Previously, this demo was virtually unplayable, since when moving between widely-spaced "clusters" of flowers, an expensive A-star search was invoked at almost every tick. Now, the vast majority of moves utilize the cache, and the demo exhibits minimal stuttering (e.g. when a single A-star search is performed when moving between distant clusters).
scripts/play.sh -i scenarios/Fun/horton.yaml --autoplay --speed 7
### Event log
An event log specific to the path cache is maintained with its own ring buffer:
scripts/play.sh -i scenarios/Testing/1569-pathfinding-cache/1569-harvest-batch.yaml --autoplay
and view http://localhost:5357/paths/log
* Highlight occurrences of structures in markdown in red
* Add a new `blueprint` entity to provide `floorplan` and `structure` commands
* new `floorplan` command to get the dimensions of a structure
* structure location is southwest instead of northwest corner
* handle structures with "holes" for statically-recognized structures and when modifying interior cells post-recognition
Each of these is its own commit for reviewability's sake.
Closes#1575
Implements structure recognition.
## Features
* Structure browsing dialog (`F6`) that becomes available if a scenario declares any recognizable structures
* Automatically recognizes statically-placed structures upon scenario initialization, accounting for occlusion by other entity/structure placement
* New `structure` command for querying location of recognized structures (primarily intended for system robots and goal checking)
* Efficiently recognizes structures immediately upon completion
* Accounts for removal of structures
* Several new integration tests
* Structured web-interface log to help understand/debug the recognition process
* Re-uses much of the functionality built previously for defining structures (#1332)
Other changes:
* Improved validation for static structure placement (ensure valid structure names instead of failing silently)
* Moved a few functions (`getNeighborLocs`, `zoomWorld`, `entityAt`, `robotWithID`, `robotWithName`) out of `Step.Util` and into `State` so that recognizer initialization, which becomes a field in `GameState`, could use them
* split `scenarioToGameState` into pure and non-pure functions
## Optimizations
Scenarios that do not make use of structure recognition are entirely unaffected, performance-wise.
Some optimizations include:
* Structure definitions must "opt-in" to participate in automatic recognition
* Aho-Corasick automatons optimized by:
* only initiate structure search if a placed entity exists on a participating structure template
* initializing different automatons for each type of "placed entity"
* pruning inapplicable row candidates for 2-D search
The row-level structure recognition cache described in #1575 was not implemented; it's probably not worth the complexity cost.
# UI Demo
scripts/play.sh -i scenarios/Testing/1575-structure-recognizer/1575-browse-structures.yaml --autoplay
1. Press `F6` for Structure Browser dialog
2. View http://localhost:5357/recognize/log and http://localhost:5357/recognize/found
# Future improvements
* Refactor `State.hs` so that the new helper functions can live elsewhere
* Support non-rectangular recognizable structures
* Allow flip/rotate of recognizable structures
* Structure ownership by robots
* Consolidate code between the Goals and Structures modal dialogs, and the Achievements browser
* Enforce minimum/maximum dimensions for structure definitions
Closes#1436.
The schema `.json` files are now the authoritative source of truth for documentation.
Wrote a very simple parser for JsonSchema to extract the documentation from JSON.
Split the README.md into [static](c314cc50a1/data/scenarios/README.md) and [auto-generated](c314cc50a1/data/scenarios/doc-fragments/SCHEMA.md) parts.
Added a custom `"footers"` key to schema files to support inclusion of other markdown files for each object description.
# Schema doc regeneration
./scripts/regenerate-schema-docs.sh
Closes#1461. Errors which used to be displayed in a pop-up window (parse errors, type errors) are now displayed inline in the REPL window instead.
- Get rid of the pop-up error dialog
- Also fix a bug introduced in #1481 where the REPL history would not be properly cleared when first starting a new scenario, because the old cache was still being used
Closes#1428
Since the authoritative validation of scenario files is actually performed by virtue of `swarm` parsing them, this CI job actually exists to ensure the JSON Schema descriptions are accurate. This is important for two purposes:
* Documentation is generated from the JSON Schema files (#1436)
* JSON Schema has integration with VS Code and other IDEs
# Testing
Verified that the schema checker action does indeed work by intentionally pushing an invalid scenario file in f789f81.
- improve layout of terms:
- break lines on binds, unless it fits on one line
- lambdas go on same line, but the body _can_ go to next line
- def and let can have long body on next line (indented for def)
- parens and braces have body indented if it does not fit on line
- closes#11
Example using `--format` from #1459:
```
> cabal run swarm -O0 -- format scenarios/Challenges/_blender/patrol-clockwise.sw
def forever = \c. c; force forever c end;
def encircle = \lDir. \rDir.
turn lDir;
b <- blocked;
if b {turn rDir} {wait 1};
fwBlocked <- blocked;
if fwBlocked {turn rDir} {move}
end;
def patrolCW = force forever (force encircle right left) end;
force patrolCW
```
Closes#1430.
# Demo
scripts/play.sh -i data/scenarios/Testing/1430-built-robot-ownership.yaml
Before, pressing `F2` would show two robots: the `base` and the system-built robot.
Now, only the `base` is shown, as the built robot is properly classified as another system robot.
DSL for programming worlds, towards #1320 and #29 (and, indirectly, toward #50, since the world DSL should make a nice target for world saves) . Eventually this should be able to recreate all the world description/building features we have, though there is still a long way to go. But currently we can at least recreate the "classic" procedurally-generated world. I think this is a solid foundation we can merge as a first step, and then work on adding more features in subsequent PRs. Below are some notes that should help in reviewing. Note that the large number of files changed is due in large part to the elimination of the `default` field in scenario descriptions; see the "changed files" section below for an overview of the important/interesting changes.
Issues split off from this one: #1394#1395#1396#1397
Major changes
============
- New `data/worlds` subdirectory
- All `.world` files are parsed at load time and saved in a `WorldMap` which gets threaded through, similar to `EntityMap` (perhaps we should think about passing around a single record instead)
- Standard "classic" world
- Used to be `testWorld2`, defined in Haskell code; now it is defined via the DSL in `worlds/classic.world`. This should make it much easier to experiment with variations.
- We can now automatically extract entities mentioned in a world DSL term with `extractEntities`. There used to be an explicit list in `testWorld2Entities`, used to check pedagogy, generate documentation, etc., but it turns out it had (predictably) gotten out of date! This can't happen anymore.
- It is now referenced in several tutorials (backstory, farming, world101, speedruns, etc.)
- The `default` field of world descriptions is no more: one can use `dsl` to just specify a constant
- Note in `Swarm.Game.State`, `dslWF` and `arrayWF` are combined using the `Monoid` instance to create `wf`.
- `Erasable`
- It used to be the case that if some kind of default terrain + entity was specified (e.g. stone + water), any `map` would completely override the default. However, we want to move towards combining everything with a `Monoid` instance. But by default this means the default entity would show through anywhere the `map` did not specify an entity. So we need a way to explicitly "erase" an entity from a lower layer.
- If `e` is a `Semigroup`, then `Maybe e` is a `Monoid` where `Nothing` acts as an identity element. Likewise, `Erasable e` is a `Monoid` but adds two new elements: `ENothing` to be an identity, and `EErase` to be an *annihilator*. i.e. combining with `EErase` is like multiplying by zero.
- We can now specify `erase` as an entity to override entity underneath.
- There are several Haskell files with only changes related to `Erasable`, relating to e.g. the world editor, `PCells`, etc.; I'm not 100% sure I've always done the right thing here.
DSL overview
===========
- Integer, float, and Boolean literals. Note that `3` is *always* an `int`, and `3.0` is a `float`. It makes things much easier to not have to deal with `3` possibly being either `int` or `float`, though it does make things slightly more annoying for programmers.
- Standard boolean, arithmetic, and comparison operators
- `if ... then ... else ...`
- `<>` operator for combining via `Semigroup` instance
- Cell literals are enclosed in curly braces. Unlike the previous awkward world description syntax with one, two, or three-element lists denoting terrain, terrain + entity, or terrain + entity + robot, there can now be any number of elements in any order.
- `{foo}` will be resolved as either terrain, an entity, or a robot, whichever is successful. So if the names are unambiguous one can just write `{tree}` or `{stone}`.
- It is possible to explicitly indicate the type of cell value with syntax like `{entity: tree}` or `{terrain: stone}`.
- Multiple items separated by commas is syntax sugar for combining with `<>`. e.g. `{tree, entity: boulder, stone} = {tree} <> {entity: boulder} <> {stone}`.
- Ability to refer to the `seed`
- Refer to the current `x` or `y` coordinates or the `hash` of the current coordinates
- `let`-expressions for multiple variables: `let x1 = e1, x2 = e2, ... in ...`
- `overlay [e1, e2, ...]` layers `e1` on the bottom, `e2` on top of that, etc., using the `Semigroup` instance for world functions
- `"foo"` imports the DSL term in `worlds/foo.world`
- `perlin` function to generate perlin noise
- `mask` function to mask with a condition
Changed files
===========
- `Swarm.Util`: moved the `acquire` function here and gave it a more descriptive name.
- `Swarm.Doc.Gen`: can now extract mentioned entities directly.
- `Swarm.Game.Failure`: added new failure modes
- `Swarm.Game.Scenario.Topography.WorldDescription`: get rid of `defaultTerrain` field, add `worldProg` for DSL.
- `Swarm.Game.State`: see comment.
- `Swarm.Game.World`: a bit of reorganization. Added a bunch of modules under this.
- `Swarm.Game.World.Coords`: moved some code here from `Swarm.Game.World`.
- `Swarm.Game.World.Gen`: moved some things here from `Swarm.Game.WorldGen` (also deleted a bunch of irrelevant code), and also added the `extractEntities` function to get all entities mentioned by a DSL term.
- `Swarm.Game.World.Syntax`: raw, untyped syntax for world DSL terms.
- `Swarm.Game.World.Parse`: parser for world DSL terms. Fairly standard.
- `Swarm.Game.World.Typecheck`: takes raw, untyped terms produced by the parser and both typechecks and elaborates them into a simpler core language. An interesting feature is that the core language is *type-indexed*, so that the Haskell type system is actually ensuring that our typechecker is correct; every typechecked world DSL term value has a type which is indexed by a Haskell type corresponding to the type of the underlying DSL term. For example, `{entity: tree}` would have a type like `TTerm [] (World CellVall)` etc. Once terms make it through the typechecker, there cannot possibly be any bugs in the rest of the pipeline which would result in a crash, because the Haskell type system. (There could of course be *semantic* bugs.) Understanding exactly how the typechecker works is not too important. Of interest may be the `resolveCell` function, which determines how we decide what `Cell` is represented by a cell expression in curly braces.
- `Swarm.Game.World.Abstract`: compile elaborated, typechecked world DSL terms down into an extremely simple core language with only constants and function application. This gives us very fast evaluation of world DSL terms. Understanding this module is not really necessary but there is a link to a blog post for those who are interested in how it works.
- `Swarm.Game.World.Compile`: a further processing/compilation step after `Swarm.Game.World.Abstract`. Currently we don't actually use this, since it doesn't seem like it makes a big efficiency difference.
- `Swarm.Game.World.Interpret`: interpreter for abstracted world DSL terms.
- `Swarm.Game.World.Eval`: just puts together the pieces of the pipeline to evaluate a typechecked world DSL term.
- `Swarm.Game.World.Load`: just loading world DSL terms from disk.
* use `Markdown.Document` as `entityDescription`
* add missing spaces in `chunksOf`
* fix code in `entities.yaml` (mostly types and few outdated snippets)
* add code markdown in craft tutorial
* use colours for types and entities
- closes#1408
- closes#1409
