This needed a bit more surgery than one would like, since we need to
translate the indirect jump to an if-else chain, which means creating a
bunch of "extra" blocks from within the `CrucGen` monad.
This reverts commit 86ef62645d.
This commit generates invalid Crucible CFGs, which causes downstream analysis to
fail. Reverting the commit will allow downstream clients to discard invalid CFGs.
This means far fewer instructions (and hence fewer registers), and in
turn a lot less heap space. Peak memory usage is cut in half running
Brittle on a PPC64 exe with standard library.
Generating the type of the register structure on demand was causing
`TypeRepr` to be the biggest chunk of the heap. Similarly, we only need
to create a new `Position` when we change the offset.
Most crucially, make the `CrucGen` monad itself strict. The heap was
filling up with old `CrucGenState`s being held onto by unevaluated
computations, since *every* computation was lazy. Plugged a few other
sources of `CrucGenState` leaks as well.
This helps to immunize against changes in SimContext... e.g. the
addition on the profilingMetrics field that initSimContext provides a
default value for.
The llvm memory model was extended with better diagnostics and configurable
handling of undefined behavior. macaw-symbolic uses no undefined behavior
checking, as those operations are only undefined in C.
Before, we just discarded them during the translation. They are useful metadata
for generating diagnostics in Crucible, so this commit translates them. They
are no-ops during symbolic evaluation.
To make them truly useful, they need to include the address of the block that
they belong to (their data payload in macaw is just an offset from the start of
a block). This information wasn't available before, so it has to be plumbed
through in macaw-x86.
This cleanup consolidates the interface to macaw symbolic into two (and a half)
modules:
- Data.Macaw.Symbolic for clients who just need to symbolically simulate
machine code
- Data.Macaw.Symbolic.Backend for clients that need to implement new
architectures
- Data.Macaw.Symbolic.Memory provides a reusable example implementation of
machine pointer to LLVM memory model pointer mapping
Most functions are now documented and are grouped by use case. There are two
worked (compiling) examples in the haddocks that show how to translate Macaw
into Crucible and then symbolically simulate the results (including setting up
all aspects of Crucible). The examples are included in the symbolic/examples
directory and can be loaded with GHCi to type check them.
The Data.Macaw.Symbolic.Memory module still needs a worked example.
There were very few changes to actual code as part of this overhaul, but there
are a few places where complicated functions were hidden behind newtypes, as
users never need to construct the values themselves (e.g., MacawArchEvalFn and
MacawSymbolicArchFunctions). There was also a slight consolidation of
constraint synonyms to reduce duplication. All callers will have to be updated.
There is also now a README for macaw-symbolic that explains its purpose and
includes pointers to the new haddocks.
This commit also fixes up the (minor) breakage in the macaw-x86-symbolic
implementation from the API changes.
We were getting "Unterminated crucible block" errors for any code
containing the X86 HLT instruction. An ErrorStmt is perhaps not
precisely what HLT means, but we're going with that for the moment.
This update renames many of the declarations exported by
Data.Macaw.Memory so that we have more consistent names.
The majority of the existing names are now exported with DEPRECATION
warnings. Some of the symbol declarations that were not used by the
Memory datatype have been moved to other modules.
The minor version of macaw-base has been incremented.
Now one can either directly produce an SSA CFG or produce a registerized
one, perhaps mess with it (as with the new
`Lang.Crucible.Utils.RegRewrite` module), then translate it to SSA.
It needs to take (and return) a Crucible state so that we can insert the new
function handle into the handle map (so that the Crucible Call statement can
find it).
The GlobalMap is mapping from virtual addresses computed by a program to the
corresponding logical address in the LLVM memory model during symbolic
simulation. It is needed because addresses in binaries are computed from
bitvectors, which are not valid pointers in the LLVM memory model.
This change turns the GlobalMap from a Data.Map into a function, which is more
flexible and allows for a wider range of possible implementations of this
functionality, especially implementations that introduce numerous disjoint
segments for the original binary contents.
The former strategy was to represent macaw calls using a macaw-specific
MacawCall statement, which was interpreted by a call handler (which took
registers+memory as input and produced new registers+memory as an output). This
worked for cases where the callee had a summary, but did not allow for
simulating the called function inline. Moreover, the OverrideSim monad doesn't
admit recursive calls in this context (we can make the call, but we can't get
the final simulator state out, which we would need to implement a call handler
in macaw-symbolic).
The new strategy is to translate macaw calls into two separate statements:
1. A `LookupFunctionHandle` call, which returns a Crucible FunctionHandle, and
2. A normal Crucible `Call`
The interpretation of LookupFunctionHandle has the full register+memory state
available, and can inspect the IP to determine which function has been
called (and provide the necessary FunctionHandle, which will be interpreted by
Crucible in the standard way). Note that the handler is in IO, so client code
can translate functions being simulated into Crucible on-demand.
This commit introduces a new syntax extension for the macaw translation to
represent the ArchState statement: MacawArchStateUpdate.
Also adds some new instances for MacawCrucibleValue.
The new statement is called `ArchState`, and has two fields: an address and a
map. The address is the address of the instruction it is standing in for. The
map contains a mapping from the *machine registers* that the instruction updated
to the *macaw values* that were assigned to those locations.
This is useful metadata for debugging, but is also required to do some types of
architecture-independent analysis (where we can still reason about machine
register contents).
When computing pointers we don't always check that the results are valid.
Instead, we do the check whenever we use the pointers.
The reason is to support code where pointers are temporarily "bad"
but are never used that way. For example:
subq $10, %aex # aex contains a pointer
Loop:
addq $10, %aex
...
We don't really do anything with alignment, but sometime asm code
ands pointers to align them. For example `andq $(-64), %rsp`
aligns the pointer to a multiple of 64.
To support code like this we treat "and"-ing a pointer with a special
constant of the form 0xFFFF...FF000 (i.e., and alignment) as a subtracting
`0x0000...00XXX` where the `XXX` is symbolic.
This looses some information (i.e., we don't know that the result is aligned).
However, it is good enough for checking memory safety, as it covers
all possible results of the alignment.
Previously we were asserting that some bogus-y things don't happen.
Unfortunately, these expressions can occur in code that was not
directly written by the user (e.g., comparisons for setting various
machine flags). To allow for that, we allow the expressions, but
give them undefined values. So the proof will succeed only if it
does not depend on the values of these bogus comparisons.
We basically punt, by passing-in a function to use as the implementation of
all functions. This function is supposed to look at the IP, and
decide what to do.
