If a test times out, that means it hasn't returned a prompt within
reasonable time. Continuing to type at it is unlikely to be helpful,
since the lack of a prompt indicates that the subprocess has hung, so
subsequent tests are likely to time out as well.
This is particularly annoying in step 5, where the line that usually
times out, "(def! res2 (sum2 10000 0))" has its output ignored, so a
timeout is treated as success, meaning that runtest proceeds to the next
test, which of course also times out.
Implementations don't generally deliberately emit CR, so any CR we see
was introduced either by the host language runtime or the OS terminal
driver, neither of which we're trying to test.
Some of the complexity of runtest is in the way that it insists on
checking whether the implementation managed to echo its input line
correctly. This isn't really an important part of the implementation,
though, and when using friendly line-editing libraries the echoing can
often end up containing escape sequences. Simplify things by just
assuming that anything up to the first newline is our input being echoed
back.
This also means that various pre-processing of input is no longer
required.
Be more consistent about what FDs the executing implementation will
have during execution. This makes it more consistent with behavior of
launching the implementation directly.
The interop tests weren't updated when regex support was added to
runtest for matching output data. This was causing a bunch of
implementations to silently fail while testing interop. Fix runtest.py
to fail on bad test data and fix 21 implementations with the old style
output matches.
This allows carriage returns to be specified in tests which allows
multiline tests to be written i.e. by using Ctrl-V Enter to put
a literal carriage return (^M) into the test definition. The effect of
this is to send a single string to the REPL that contains newlines
which normally can't be done because readline behavior will split it
into two separate lines.
- Truncate ruby and python stacktraces since they can get very long
in certain cases causing problems.
- Fix Clojure West example with proper escaping.
- Add read-file which is similar to read-string but from a file name
rather than a string. This allows steps 0-2 to load although each
one eventuall crashes with out of memory after evaluating "123"
a few times.
- basicpp:
- Renumber the line numbers so they are ordinally increasing. This
saves 150 or so bytes because GOTO/GOSUB calls have smaller line
numbers.
- Shrink 'IF 123' -> 'IF123' for almost 300 byte savings.[:w
- Simplify PR_MEMORY_SUMMARY output. Save 75 bytes
- Add missing runtest.py change that allows basic tests to pass.
- tests/*: marked with deferable and optional runtest flags. Some
moving around of tests within different sections to more closely
align with the guide.
- runtest.py has --deferable/--no-deferable and
--optional/--no-optional to control whether tests marked as
deferable and optional are tested.
- Makefile: DEFERABLE and OPTIONAL top level flags to be passed to
runtest when running tests.
- Also, fix problems in java quoting revealed by step7 test changes.
- make sure readline has large enough string for steps2-A
- fix keyword? so that it doesn't leave anything extra on the stack
- fix argparse command line parsing so that that the second '--' on
the command line is not dropped.
Fixes: https://github.com/kanaka/mal/issues/40
- Simplify exception handling by combining keyboard exception into
other exception handler. Print out output buffer on all exceptions.
It is an ugly fix, but basically, use regular Popen (without pty) when
using mono, and do hardcoded echo of the commands when they are
sent/written. It could be worse I suppose...
