macaw/macaw-ppc-symbolic/tests
Brett Boston a336895da7
Add optional override for MacawArchStmtExtensions to genArchVals (#230)
This change adds an optional argument to `genArchVals` that allows client code to override the backend translation behavior of `MacawArchStmtExtension`s on a statement-by-statement basis.  The new argument has type `Maybe (MacawArchStmtExtensionOverride arch)`, where `MacawArchStmtExtensionOverride` is a function that takes a statement and a crucible state, and returns an optional tuple containing the value produced by the statement, as well as an updated state.  Returning 'Nothing' indicates that the backend should use its default handler for the statement.

Client code that wishes to maintain the existing default behavior in all cases can simply pass `Nothing` for the new argument to `genArchVals`.
2021-09-14 18:24:47 -07:00
..
fail Add a testing framework for macaw-symbolic (#184) 2021-03-01 09:21:44 -08:00
pass Add a testing framework for macaw-symbolic (#184) 2021-03-01 09:21:44 -08:00
Main.hs Add optional override for MacawArchStmtExtensions to genArchVals (#230) 2021-09-14 18:24:47 -07:00
README.org Add a testing framework for macaw-symbolic (#184) 2021-03-01 09:21:44 -08:00

Overview

This test suite tests that symbolic execution of PowerPC programs works. It is also a convenient place to test the semantics. The test harness looks for binaries in the pass and fail subdirectories. In both cases, it enumerates the functions whose names are prefixed by test_, each of which are symbolically executed with fully symbolic initial states. It treats the return value of each test function as an assertion that it will attempt to prove (i.e., it checks the satisfiability of the assertion). Test cases in pass are expected to return a value that is always true, while the test cases in fail are expected to not always be true (but note that the precise models are not inspected). The test harness uses yices for path satisfiability checking and z3 to prove goals. Since path satisfiability checking is enabled, test cases can include loops as long as they symbolically terminate.

Usage

The test runner has two command line options (beyond the defaults for tasty):

  • --save-macaw: Saves the Macaw IR for each test case to /tmp for debugging purposes
  • --save-smt: Saves the generated SMTLib for each test to /tmp for debugging purposes

Limitations

  • It currently tests both optimized an unoptimized binaries. It is intended that this set will expand and that a wide variety of compilers will be tested.
  • Only two solvers are involved in the test, rather than a whole matrix
  • Function calls are not supported in test cases
  • There are no facilities for generating symbolic data beyond the initial symbolic arguments to each function