This would come in handy for an application where I wish to pass a
`NonEmptyVector` to `newMergedGlobalMemoryWith`. Currently, I have to convert
the `NonEmptyVector` to a `NonEmpty` list to accomplish this, wish seems
wasteful given that `newMergedGlobalMemoryWith` only needs to use the
`Foldable` interface.
The return address gets masked and has the low-bit set in an obtuse way due to
the semantics. This threw off the call detection.
This change matches against the quirky pattern.
This change adds support for RV32GC RISCV binaries. Specifically, it:
* Updates the return matcher to recognize returns in 32-bit binaries
* Updates detection of unsupported binaries to allow RV32GC binaries
* Adds RV32GC versions of the RV64GC tests
This change adds a function `newMergedGlobalMemoryWith`, which acts like
`newGlobalMemoryWith` but takes a list of macaw memories and merges them
into a flat address space. This aids in reasoning dynamically linked
programs.
Before, the API provided by macaw-symbolic asserted the initial value of each byte of memory individually. This was fairly expensive for large binaries, as each such assertion flushed the solver pipe.
This change generates a large conjunction of assertions and sends them all at once. In unscientific testing, this saved half an hour on a large binary.
API Changes:
- Note that it introduces a minor API change. The optimization required that the `sym` parameter be concretely an `ExprBuilder`.
* riscv: added grift as submodule
* added macaw-riscv project
* make arch polykinded everywhere in macaw base
* stubbed out riscv_info
* update grift
* started on RISCVReg
* started on RISCVReg
* RegisterInfo instance for RISCVReg (a few unimplemented fields)
* filled out archRegSet
* filled out withArchConstraints, archAddrWidth, and archEndianness
* added Arch module
* RISCV initialBlockRegs
* preliminary work on disassembleFn
* wip: disassembleFn
* made things more lens-y
* wip: disassemble instruction
* finished disassembly of grift assignment statements
* separated out DisInstM into separate module
* disassembly wip
* finished disassembleBlock
* Finished riscvDisassembleFn
* bump grift submodule
* made macaw discovery poly-kinded
* added risc-v test suite
* added risc-v test suite
* fixed macaw semantics to hardware x0 to constant value 0
* added riscvPreserveReg based on assembler's manual
* riscvDemandContext
* successfully disassembled a block!
* enhanced tests to allow optional entry point spec
* stubbed out identifyCall
* stubbed out identifyReturn
* passing initial test
* added checkForReturnAddr stub
* fleshed out identifyCall and identifyReturn
* update grift submodule
* bug fix and exception handling
* added EXC register, which tracks whether or not we've attempted to
read from/write to any system registers.
* Replaced custom CSR type with GRIFT's (but we're not using it
currently)
* added better show instance for GPRs we should migrate this to a
GRIFT pretty printer at some point)
* Fixed a vicious bug in the semantics; unsigned and signed LT were
getting swapped in translation
* added pattern synonyms for GPRs
* improved docs and fixed RISCVReg bug (GP was 3 instead of 4)
* changed undefineds to errors
* changed RISCV class to RISCVConstraints
* wrapped GRIFT's "RV" parameter in a type to remove the need to make
macaw architecture parameter polykinded
* rolled back all changes to macaw base that made things poly-kinded
* reverted two more macaw core changes, updated license, removed old PPC test
* macaw: update to upstream changes in bv-sized and grift
* address code review comments
* macaw-riscv: expose fewer modules
* Update RISCVTermStmt definition
* Update riscv_info. macaw-riscv now builds against master
* Update bv-sized and cabal freeze files
* Update cabal freeze files with satisfying lens version
* Get tests building
* Fix printf runtime error
* Add simpler tests
* Change RISCV target version and update grift pointer
[skip ci]
* Compressed branch test passes
[skip ci]
* Add additional small tests
[skip ci]
* Introduce a syscall PrimFn
* Syscalls now correctly classified
* Fix return regs from syscall
* Extract syscall arguments
* Update expected riscv test results
* Add macaw-riscv build + test to CI
* Get building with GHC 9.0.2
* Revert "Update cabal freeze files with satisfying lens version"
This reverts commit 4aa95c19c3.
* Install softfloat in CI
* Update Grift
* Some initial cleanup
* More cleanup
* Resolve FIXME on getReg
* Detect and only accept rv64gc rvreprs
* Address Tristan's PR comments
* Update Grift pointer
* Add info on installing Softfloat to README for macaw-riscv
* Add missing submodule step to softfloat build instructions
Co-authored-by: Ben Selfridge <benselfridge@000279.local>
Co-authored-by: Valentin Robert <val@galois.com>
* Translate PLTStubs as tail calls
This change modifies `addMacawParsedTermStmt` to translate `PLTStub`s as
tail calls.
* Replace CR.Call + CR.Return with CR.TailCall
* Add support for standalone PIEs
This changeset adds support for standalone position independent
executables (PIEs) that do not make use of procedure linkage tables. It
does so by adding relative address support to `populateSegmentChunk` and
adding an additional simplification rule for Aarch32.
This covers part of the work for #234.
* Remove NoOp + replace mult with left shift
This mostly deals with the splitting of the old `sym` type into
two: one for dealing with expression creation, and a new simulator
backend type for dealing with control-flow and assertions.
See the writeup in Crucible.hs in this commit for details. In short, the recent
changes to generalize `PtrAdd` triggered a failing proof obligation due to a use
of `llvmPointer_bv`. The new implementation is as sound as the previous one,
but more general.
Fixes#260
GHC 9.0 uncovered this type family as being unused. (See
https://gitlab.haskell.org/ghc/ghc/-/issues/18470, which made
`-Wunused-top-binds` more clever about detecting unused, closed
type families like `FloatInfoFromSSEType`.) Let's remove it to avoid
an `-Wunused-top-binds` warning.
While pre-9.0 versions of GHC would silently turn negative right shifts into
left shifts, GHC 9.0 will throw an `arithmetic overflow` exception instead.
This patch makes this behavior explicit in `macaw-semmc` to allow the code to
work on GHC 9.0.
Fixes#212.
This contains a variety of fixes needed to make the packages in the `macaw`
repo compile with GHC 9.0:
* GHC 9.0 implements simplified subsumption (see
[here](https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.0?version_id=5fcd0a50e0872efb3c38a32db140506da8310d87#simplified-subsumption)).
In most cases, adapting to this is a matter of manually eta expanding
definitions, such as in `base:Data.Macaw.Analysis.RegisterUse`. In the case
of `macaw-x86-symbolic:Data.Macaw.X86.Crucible`, the type signature of
`evalExt` had to be made more specific to adapt to the loss of contravariance
when typechecking `(->)`.
* GHC's constraint solver now solves constraints in each top-level group
sooner (see
[here](https://gitlab.haskell.org/ghc/ghc/-/wikis/migration/9.0?version_id=5fcd0a50e0872efb3c38a32db140506da8310d87#the-order-of-th-splices-is-more-important)).
This affects `macaw-aarch32` and `macaw-symbolic`, as they separate top-level
groups with `$(return [])` Template Haskell splices. The previous locations
of these splices made it so that the TH-generated instances in that package
were not available to any code before the splice, resulting in type errors
when compiled with GHC 9.0.
To overcome this, I rearranged the TH-generated instances so that they appear
before the top-level groups that make use of them.
* GHC 9.0 now enables `-Wstar-is-type` in `-Wall`, so this patch replaces some
uses of `*` with `Data.Kind.Type`. `Data.Kind` requires the use of GHC 8.0 or
later, so this patch also updates thes lower bounds on `base` to `>= 4.9` in
the appropriate `.cabal` files. (I'm fairly certain that this requirement was
already present implicity, but better to be explicit about it.)
* The `asl-translator`, `crucible`, and `semmc` submodules were updated to
allow them to build with GHC 9.0. The `llvm-pretty` and
`llvm-pretty-bc-parser` submodules were also bumped to accommodate unrelated
changes in `crucible` that were brought in.
* The upper version bounds on `doctest` in `macaw-symbolic`'s test suite were
raised to allow it to build with GHC 9.0.
There were two identical definitions of `toCrucibleEndian`, one in
`D.M.S.Memory` and another in `D.M.S.Testing`. This commit removes the
latter in favor of the former, which is actually exported.
Have all additions (at any bit width) go through the special PtrAdd
handler (rather than BVAdd). Also add special handlers for truncation and
extension.
These changes support architectures that do pointer operations at non-pointer
widths (e.g., to detect overflow). These new operations apply the named
operations over just the offset of pointers, preserving the block id.
The `Data.Macaw.Symbolic.Memory` module provides a default memory model and
initial memory setup that is suitable for many symbolic execution
workloads. However, the defaults cannot handle dynamically-linked programs, as
it calls `error` when it attempts to determine an initial value for relocations
it finds in memory.
There are no good defaults for this, as what those values should be depend a lot
on what the verifier wants to prove.
This commit adds some hooks to configure this behavior in the verifier, and is
designed to be extensible and enable other configuration choices where
reasonable.
The original API is unchanged, as it calls the added `newGlobalMemoryWith`
function with a default set of hooks. Callers with special memory handling
needs are directed to use that function.
Add support for abstract interpretation of division operations in the ARM
backend (when the operands to division are concretely known).
This commit also adds extended documentation on the semantics of these
operations.
The concrete evaluation eliminates constant division operations. The abstract
cases are probably obsolete in light of that, but are still interesting...
These changes were motivated by insufficient simplification around some of the syscall/errno handling code in musl
The old formulation (with system calls as block terminators) proved to be
impossible to implement properly. Handlers for syntax overrides have very
limited types (`IO`, rather than `OverrideSim`), which made symbolic branching
and reusing overrides impossible.
This change replaces the system call block terminator with an arch-specific
function that is translated into a function handle lookup (which is then
dispatched to with a call).
Unfortunately, this refactoring required combining the AArch32 simplification
module with the architecture extension definitions, due to the new translation
relying on the simplifier instance.
This updates the `crucible` submodule to include GaloisInc/crucible#906
(`Control granularity of reading uninitialized memory`), as well as the
`semmc` submodule to bring in corresponding changes on its side
(GaloisInc/semmc#69). Some additional `?memOpts :: MemOptions` constraints
needed to be added to some functions in `macaw-symbolic` and
`macaw-refinement` as a result.
This updates the `llvm-pretty-bc-parser` to include these PRs:
* GaloisInc/llvm-pretty-bc-parser#159 (`Support parsing fneg instructions`)
* GaloisInc/llvm-pretty-bc-parser#166 (`Support parsing freeze instructions`)
* GaloisInc/llvm-pretty-bc-parser#164 (`Parse DebugInfoEnumerator properly on LLVM 12+`)
* GaloisInc/llvm-pretty-bc-parser#162 (`Support parsing dict{Associated,Allocated,Rank} fields introduced in LLVM 12`)
The `llvm-pretty` submodule had corresponding changes as well. These were
included as part of a previous commit to update the `llvm-pretty` submodule
(952fe5578d), but the `llvm-pretty-bc-parser`
submodule was not updated at the same time. This commit brings the two back
into harmony.
Adds support in macaw-aarch32 for conditional returns. These are not supported in core macaw, and are thus architecture-specific block terminators.
This required changes to the type of arch-specific block terminators. Before, `ArchTermStmt` was only parameterized by a state thread (`ids`). This meant that they could not contain macaw (or crucible) values. Some work on. AArch32 requires being able to store condition values in arch terminators (to support conditional returns). This change modifies the `ArchTermStmt` to enable this, which requires a bit of plumbing through various definitions and some extra instances.
In support of actually using this, it also became necessary to plumb fallthrough block labels through the architecture-specific terminator translation in macaw-symbolic.
Note that this change was overdue, as the PowerPC backend was storing macaw values in a way that would have rendered them unusable in the macaw-ppc-symbolic translation, had any interpretation been provided. These new changes will enable a handler to be written for the conditional PowerPC trap instructions.
PowerPC, x86, and ARM have been updated.
Improves the macaw-aarch32 tests. There is now a command line option to save the generated macaw IR for each
discovered function to /tmp. Note that this reuses some infrastructure from the macaw-symbolic tests. This
shared functionality should be extracted into a macaw-testing library.
This change makes the block classifier heuristic part of the `ArchitectureInfo`
structure. This enables clients and architecture backends to customize the
block classification heuristics. This is most useful for architectures that
have complex architecture-specific block terminators that require analysis to
generate (e.g., conditional returns). It will also make macaw-refinement
simpler in the future, as the SMT-based refinement is just an additional block
classifier (but is currently implemented via a hacky side channel).
This change introduces an ancillary change, which should not be very
user-visible.
It splits the Macaw.Discovery and Macaw.Discovery.State modules to break
module import cycles in a way that enables us to expose the classifier. This
should not be user-visible, as Macaw.Discovery still exports the same
names (with one minor exception that should not appear in user code).
It also moves the definition of the `ArchBlockPrecond` type family; the few
affected places should be updated. User code should probably not be able to see
this.
See `Note [Sign-extending immediate operands in push]` in
`Data.Macaw.X86.Semantics` for the full story. I have also added a test case
in `macaw-x86-symbolic` which ensures that the stack-pointer-decrementing
logic behaves as one would expect.
Bumps in the `flexdis86` submodule to bring in GaloisInc/flexdis86#37.
Fixes#235.
It is possible for later `ArchState` updates to reference the `AssignId` of
the `AssignStmt` that is dropped as a part of `stripPLTRead`, so make sure to
prune such updates from the `ArchState` to avoid referencing the now
out-of-scope `AssignId`.
This change adds an optional argument to `genArchVals` that allows client code to override the backend translation behavior of `MacawArchStmtExtension`s on a statement-by-statement basis. The new argument has type `Maybe (MacawArchStmtExtensionOverride arch)`, where `MacawArchStmtExtensionOverride` is a function that takes a statement and a crucible state, and returns an optional tuple containing the value produced by the statement, as well as an updated state. Returning 'Nothing' indicates that the backend should use its default handler for the statement.
Client code that wishes to maintain the existing default behavior in all cases can simply pass `Nothing` for the new argument to `genArchVals`.
Implement support for symbolically executing system calls in macaw-symbolic. **To update code that does not need to symbolically execute system calls (i.e., most clients of macaw-symbolic), just pass the new `unsupportedSyscalls` default handler as the fifth argument of `macawExtensions`.**
The primary interface is via the new `LookupSyscallHandle` callback passed to `macawExtensions`. This callback inspects the environment and returns a Crucible `FunctionHandle` that models the behavior of the requested system call. Note that this mechanism only supports concrete system calls (i.e., system calls where the system call number is concrete). The x86 backend has been updated to support this new functionality.
The representation of system calls in macaw is still architecture-specific (because there are interesting differences between system call instructions across architectures). The idea is that system calls are now treated in two steps:
1. A macaw-symbolic extension statement that looks up the override to invoke for the given syscall (returned as a Crucible FunctionHandle)
2. A call to that handle
We need this two step approach because the handlers that interpret syntax extension statements cannot symbolically branch (and thus cannot call overrides). The extension interpreter just looks up the necessary handle and uses the standard call/override machinery to handle any branching required to support the system call model functionality.
The major complication to this approach is that system calls need to update values in registers when they return. To capture these updates, the architecture-specific syntax extension needs to explicitly update any machine registers that could possibly be affected. The explicit updates are necessary because machine registers do not exist anymore at the macaw-symbolic level (at least within a block). To handle all of these constraints:
1. System calls are represented as extension functions at the macaw level when lifted from machine code.
2. During translation into crucible (via macaw-symbolic), the extension functions are translated into two statements: a function handle lookup and then a function call (with the return values being explicitly threaded through the Crucible function).
3. During symbolic execution, the lookup statement examines the environment to return the necessary function handle, while the handle is called via the normal machinery.
Note that the feature is entirely controlled by the `LookupSyscallHandle` function, which determines the system call dispatch policy. No system call models are included with this change.
Co-authored-by: Brett Boston <boston@galois.com>
