This commit is contained in:
Mark Wotton 2020-08-09 11:47:32 -04:00
parent ef7915fc97
commit bb7fdefc2a
5 changed files with 86 additions and 71 deletions

View File

@ -24,3 +24,43 @@ roboservant avoids this by using
[quickcheck-state-machine](https://github.com/advancedtelematic/quickcheck-state-machine#readme),
which models the dynamic state in such a way that we can use results
of previous calls.
# concept
we start with a servant api and a server that fulfills the type.
From that api, we should be able to summon up an empty type-indexed store with a key
for each response type in the API.
We can then look at each callable endpoint, and eliminate any that require values that are
empty in the type-indexed store. this allows a generative process where we can extend a sequence
of calls indefinitely, by making a call to the concrete server and recording the result in the
type-indexed store.
## extensions
- add some "starter" values to the store
- there may be a JWT that's established outside the servant app, for instance.
- `class Extras a where extras :: Gen [a]`
- default implementation `pure []`
- selectively allow some types to create values we haven't seen from the api.
`newtype FirstName = FirstName Text`, say.
- break down each response type into its components
- if i have
- `data Foo = FBar Bar | FBaz Baz`
- an endpoint `foo` that returns a `Foo`
- and an endpoint `bar` that takes a `Bar`
- I should be able to call `foo` to get a `Foo`, and if it happens to be an `FBar Bar`, I
should be able to use that `Bar` to call `bar`.
## applications
- testing
- some properties should always hold (no 500s)
- there may be some other properties that hold contextually
- healthcheck should be 200
- test complex permissions/ownership/delegation logic - should never be able to
get access to something you don't own or haven't been delegated access to.
- benchmarking
- we can generate "big-enough" call sequences, then save the database & a sample call for each
endpoint that takes long enough to be a reasonable test.
- from this we can generate tests that a given call on that setup never gets slower.

View File

@ -21,9 +21,18 @@ description: Please see the README on GitHub at <https://github.com/gith
dependencies:
- base >= 4.7 && < 5
- servant
# - servant-server
- QuickCheck
- bytestring
- http-client
- quickcheck-state-machine
- servant >= 0.17
- servant-client >= 0.17
- servant-flatten
- servant-server >= 0.17
- string-conversions
- http-media
# test deps
- hspec
library:

View File

@ -4,7 +4,7 @@ cabal-version: 1.12
--
-- see: https://github.com/sol/hpack
--
-- hash: f12f192f6c83606ae6175fd14eb83a7860480c38629bf904dbbdf6cfd980ed14
-- hash: e066fe1fb50c5d775ba730812d8afb9db9b9d14e08b2c25ca2277a1d7205d852
name: roboservant
version: 0.1.0.0
@ -28,15 +28,25 @@ source-repository head
library
exposed-modules:
Roboservant
Roboservant.ContextualGenRequest
Roboservant.StateMachine
other-modules:
Paths_roboservant
hs-source-dirs:
src
build-depends:
base >=4.7 && <5
QuickCheck
, base >=4.7 && <5
, bytestring
, hspec
, servant
, http-client
, http-media
, quickcheck-state-machine
, servant >=0.17
, servant-client >=0.17
, servant-flatten
, servant-server >=0.17
, string-conversions
default-language: Haskell2010
test-suite roboservant-test
@ -48,9 +58,17 @@ test-suite roboservant-test
test
ghc-options: -threaded -rtsopts -with-rtsopts=-N
build-depends:
base >=4.7 && <5
QuickCheck
, base >=4.7 && <5
, bytestring
, hspec
, http-client
, http-media
, quickcheck-state-machine
, roboservant
, servant
, servant >=0.17
, servant-client >=0.17
, servant-flatten
, servant-server >=0.17
, string-conversions
default-language: Haskell2010

View File

@ -13,13 +13,17 @@
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE UndecidableInstances #-}
{-# LANGUAGE FunctionalDependencies #-}
module Roboservant where
import Control.Applicative
import GHC.TypeLits
import Servant.API
import Roboservant.ContextualGenRequest
import Roboservant.StateMachine
type family ExtractRespType (path :: *) :: * where
ExtractRespType (_ :> b) = ExtractRespType b
ExtractRespType (Verb (method :: StdMethod) (responseCode :: Nat) (contentTypes :: [*]) (respType :: *)) = respType

View File

@ -1,68 +1,12 @@
# This file was automatically generated by 'stack init'
#
# Some commonly used options have been documented as comments in this file.
# For advanced use and comprehensive documentation of the format, please see:
# https://docs.haskellstack.org/en/stable/yaml_configuration/
# Resolver to choose a 'specific' stackage snapshot or a compiler version.
# A snapshot resolver dictates the compiler version and the set of packages
# to be used for project dependencies. For example:
#
# resolver: lts-3.5
# resolver: nightly-2015-09-21
# resolver: ghc-7.10.2
#
# The location of a snapshot can be provided as a file or url. Stack assumes
# a snapshot provided as a file might change, whereas a url resource does not.
#
# resolver: ./custom-snapshot.yaml
# resolver: https://example.com/snapshots/2018-01-01.yaml
resolver: lts-15.15
# User packages to be built.
# Various formats can be used as shown in the example below.
#
# packages:
# - some-directory
# - https://example.com/foo/bar/baz-0.0.2.tar.gz
# subdirs:
# - auto-update
# - wai
packages:
- .
# Dependency packages to be pulled from upstream that are not in the resolver.
# These entries can reference officially published versions as well as
# forks / in-progress versions pinned to a git hash. For example:
#
# extra-deps:
# - acme-missiles-0.3
# - git: https://github.com/commercialhaskell/stack.git
# commit: e7b331f14bcffb8367cd58fbfc8b40ec7642100a
#
extra-deps:
- servant-flatten-0.2@sha256:276896f7c5cdec5b8f8493f6205fded0cc602d050b58fdb09a6d7c85c3bb0837,1234
# Override default flag values for local packages and extra-deps
# flags: {}
# Extra package databases containing global packages
# extra-package-dbs: []
# Control whether we use the GHC we find on the path
# system-ghc: true
#
# Require a specific version of stack, using version ranges
# require-stack-version: -any # Default
# require-stack-version: ">=2.1"
#
# Override the architecture used by stack, especially useful on Windows
# arch: i386
# arch: x86_64
#
# Extra directories used by stack for building
# extra-include-dirs: [/path/to/dir]
# extra-lib-dirs: [/path/to/dir]
#
# Allow a newer minor version of GHC than the snapshot specifies
# compiler-check: newer-minor
- quickcheck-state-machine-0.7.0@sha256:4fc4467380e35b88aab72c278856aacebbf95688059d8ad70eb82fe048df476b,4958
- markov-chain-usage-model-0.0.0@sha256:1afa95faeb9213c4d960a669190078b41b89169462b8edd910472980671ba8c0,2112
- servant-0.17
- servant-client-0.17
- servant-client-core-0.17
- servant-server-0.17