mirror of
https://github.com/unisonweb/unison.git
synced 2024-09-21 23:37:23 +03:00
polish, lots of helper functions
This commit is contained in:
parent
379d4d5188
commit
e292987fc7
@ -5,37 +5,53 @@ import EasyTest
|
||||
import Control.Monad
|
||||
import Control.Applicative
|
||||
|
||||
main = runOnly "addition" $ do
|
||||
expect (1 + 1 == 2)
|
||||
fork $ do
|
||||
ns <- [0..10] `forM` \n -> replicateM n (randomBetween (0 :: Int, 10))
|
||||
ns `forM_` \ns -> expect (reverse (reverse ns) == ns)
|
||||
scope "addition" $ expect (3 + 3 == 6)
|
||||
scope "always passes" $ do
|
||||
note "I'm running this test, even though it always passes!"
|
||||
ok -- like `pure ()`, but records a success result
|
||||
scope "failing test" $ crash "oh noes!!"
|
||||
main = runOnly "addition" $ tests
|
||||
[ expect (1 + 1 == 2)
|
||||
, fork $ do
|
||||
-- generate lists from size 0 to 10, of Ints in (0,43)
|
||||
-- shorthand: listsOf [0..10] (int' 0 43)
|
||||
ns <- [0..10] `forM` \n -> replicateM n (int' 0 43)
|
||||
ns `forM_` \ns -> expect (reverse (reverse ns) == ns)
|
||||
, scope "addition" $ expect (3 + 3 == 6)
|
||||
, scope "always passes" $ do
|
||||
note "I'm running this test, even though it always passes!"
|
||||
ok -- like `pure ()`, but records a success result
|
||||
, scope "failing test" $ crash "oh noes!!" ]
|
||||
```
|
||||
|
||||
The library is simple: you just write ordinary Haskell code in the `Test` monad, which has access to:
|
||||
The idea here is to write tests with ordinary Haskell code, with control flow explicit and under programmer control. Tests are values of type `Test a`, and `Test` forms a monad with access to:
|
||||
|
||||
* random numbers (the `random` and `randomBetween` functions)
|
||||
* repeatable randomness (the `random` and `random'` functions for random and bounded random values, or handy specialized `int`, `int'`, `double`, `double'`, etc)
|
||||
* I/O (via `liftIO`)
|
||||
* failure (via `crash`)
|
||||
* logging (via `note` or `noteScoped`)
|
||||
* hierarchically-named subcomputations which can be switched on and off (in the above code, only the `"addition"`-scoped test would be run, and we could do `run` instead if we wanted to run the whole suite)
|
||||
* failure (via `crash`, which yields a stack trace!)
|
||||
* logging (via `note`, `noteScoped`, or `note'`)
|
||||
* hierarchically-named subcomputations which can be switched on and off (in the above code, `"always passes" and "failing test"`-scoped tests would not be run, and we could do `run` instead of `runOnly` if we wanted to run the whole suite)
|
||||
* parallelism (note the `fork` which runs that subtree of the test suite in a parallel thread).
|
||||
* conjunction of tests via `MonadPlus` (the `<|>` operation runs both tests, even if the first test fails, and the `tests` function used above is just `msum`).
|
||||
|
||||
`Test` is an instance of everything through `MonadPlus` (the `<|>` operation runs both tests, even if the first test fails). You assemble `Test` values into a test suite using ordinary Haskell code, not framework magic. Notice that to generate a list of random values, we just `replicateM` and `forM` as usual. If this gets tedious... we can factor this logic out into helper functions!
|
||||
Using any or all of these capabilities, you assemble `Test` values into a "test suite" (just another `Test` value) using ordinary Haskell code, not framework magic. Notice that to generate a list of random values, we just `replicateM` and `forM` as usual. If this gets tedious... we can factor this logic out into helper functions! For instance:
|
||||
|
||||
```Haskell
|
||||
listOf :: Int -> Test a -> Test [a]
|
||||
listOf = replicateM
|
||||
|
||||
listsOf :: [Int] -> Test a -> Test [[a]]
|
||||
listsOf sizes gen = sizes `forM` \n -> listOf n gen
|
||||
|
||||
ex :: Test ()
|
||||
ex = do
|
||||
ns <- listsOf [0..100] int
|
||||
ns `forM_` \ns -> expect (reverse (reverse ns) == ns)
|
||||
```
|
||||
|
||||
This library is opinionated and might not be for everyone. But here's some of my thinking in writing it:
|
||||
|
||||
* Testing should uncomplicated, minimal friction, and ideally: FUN.
|
||||
* A lot of testing frameworks are weirdly optimized for adding lots of diagnostic information up front, as if just whatever diagnostic information you happen to think to capture will magically allow you to fix whatever bugs your tests reveal. EastTest takes the opposite approach: be lazy about adding diagnostics and labeling subexpressions, but make it trivial to reproduce failing tests without running your entire suite. If a test fails, you can easily rerun just that test, with the same random seed, and add whatever diagnostics or print statements you need to track down what's wrong.
|
||||
* A lot of testing frameworks are weirdly optimized for adding lots of diagnostic information up front, as if whatever diagnostic information you happen to think to capture will be exactly what is needed to fix whatever bugs your tests reveal. EasyTest takes the opposite approach: be lazy about adding diagnostics and labeling subexpressions, but make it trivial to reproduce failing tests without running your entire suite. If a test fails, you can easily rerun just that test, with the same random seed, and add whatever diagnostics or print statements you need to track down what's wrong.
|
||||
* Another reason not to add diagnostics up front: you avoid needing to remember two different versions of every function or operator (the one you use in your regular code, and the one you use with your testing "framework" to supply diagnostics). HUnit has operators named `(@=?)`, `(~?=)`, and a bunch of others for asserting equality with diagnostics on failure. QuickCheck has `(.&&.)` and `(.||.)`. Just... no.
|
||||
* HUnit, QuickCheck, SmallCheck, Tasty, and whatever else are frameworks that hide control flow from the programmer and make some forms of control flow difficult or impossible to specify (for instance, you can't do I/O in your QuickCheck tests!). In contrast, EasyTest is just a single data type with a monadic API and a few helper functions. You assemble your tests using ordinary monadic code, and there is never any magic. Want to abstract over something? _Write a regular function._ Need to generate some testing data? Write regular functions.
|
||||
* "How do I modify the number of generated test cases for QuickCheck for just one of my properties?" Or control the maximum size for these `Gen` and `Arbitrary` types? Some arbitrary "configuration setting" that you have to look up every time.
|
||||
* "How do I modify the number of generated test cases for QuickCheck for just one of my properties?" Or control the maximum size for these `Gen` and `Arbitrary` types? Some arbitrary "configuration setting" that you have to look up every time. No thanks.
|
||||
* Global configuration settings are evil. I want fine-grained control over the amount of parallelism, test case sizes, and so on.
|
||||
* Most of the functionality of QuickCheck is overkill anyway! There's no need for `Arbitrary` instances (explicit generation is totally fine, and even preferred in most cases), `Coarbitrary` (cute, but not useful when the HOF you are testing is parametric), or shrinking (just generate your test cases in increasing sizes, and your first failure will be the smallest).
|
||||
* Most of the functionality of QuickCheck is overkill anyway! There's no need for `Arbitrary` instances (explicit generation is totally fine, and even preferred in most cases), `Coarbitrary` (cute, but not useful when the HOF you are testing is parametric), or shrinking (just generate your test cases in increasing sizes, and your first failure will be the smallest!).
|
||||
|
||||
I hope that you enjoy the library and that it proves useful.
|
||||
|
@ -2,7 +2,7 @@
|
||||
{-# Language FunctionalDependencies #-}
|
||||
{-# Language GeneralizedNewtypeDeriving #-}
|
||||
|
||||
module EasyTest (Test, crash, currentScope, noteScoped, skip, ok, fork, fork', scope, note, expect, tests, random, randomBetween, run', runOnly, run, rerun, rerunOnly, parseMessages, module Control.Monad.IO.Class) where
|
||||
module EasyTest where
|
||||
|
||||
import Control.Applicative
|
||||
import Control.Concurrent
|
||||
@ -12,12 +12,14 @@ import Control.Monad
|
||||
import Control.Monad.IO.Class
|
||||
import Control.Monad.Reader
|
||||
import Data.List
|
||||
import Data.Map (Map)
|
||||
import Data.Word
|
||||
import GHC.Stack
|
||||
import qualified System.Random as Random
|
||||
import System.Random (Random)
|
||||
import System.Exit
|
||||
import System.Random (Random)
|
||||
import qualified Control.Concurrent.Async as A
|
||||
import qualified Data.Map as Map
|
||||
import qualified System.Random as Random
|
||||
|
||||
data Status = Failed | Passed | Skipped
|
||||
|
||||
@ -91,23 +93,28 @@ run' seed note allow (Test t) = do
|
||||
Right () -> note $ "Waiting for any asynchronously spawned tests to complete ..."
|
||||
atomically $ writeTQueue resultsQ Nothing
|
||||
_ <- A.waitCatch rs
|
||||
note line
|
||||
note "\n"
|
||||
resultsMap <- readTVarIO results
|
||||
let
|
||||
resultsList = Map.toList resultsMap
|
||||
succeeded = length [ a | a@(_, Passed) <- resultsList ]
|
||||
failures = [ a | (a, Failed) <- resultsList ]
|
||||
failed = length failures
|
||||
note $ " " ++ show succeeded ++ (if failed == 0 then " PASSED" else " passed")
|
||||
note $ " " ++ show (length failures) ++ (if failed == 0 then " failed" else " FAILED (failed scopes below)")
|
||||
case failures of
|
||||
[] -> do
|
||||
note "\n"
|
||||
note line
|
||||
note "✅ all tests passed! 👍 🎉"
|
||||
case succeeded of
|
||||
0 -> do
|
||||
note "😶 hmm ... no test results recorded"
|
||||
note "Tip: use `ok`, `expect`, or `crash` to record results"
|
||||
note "Tip: if running via `runOnly` or `rerunOnly`, check for typos"
|
||||
1 -> note $ "✅ 1 test passed, no failures! 👍 🎉"
|
||||
_ -> note $ "✅ " ++ show succeeded ++ " tests passed, no failures! 👍 🎉"
|
||||
(hd:_) -> do
|
||||
note $ " " ++ intercalate "\n " (map showMessages failures)
|
||||
note line
|
||||
note "\n"
|
||||
note $ " " ++ show succeeded ++ (if failed == 0 then " PASSED" else " passed")
|
||||
note $ " " ++ show (length failures) ++ (if failed == 0 then " failed" else " FAILED (failed scopes below)")
|
||||
note $ " " ++ intercalate "\n " (map (show . showMessages) failures)
|
||||
note ""
|
||||
note $ " To rerun with same random seed:\n"
|
||||
note $ " EasyTest.rerun " ++ show seed
|
||||
@ -142,6 +149,9 @@ note msg = do
|
||||
liftIO $ note_ msg
|
||||
pure ()
|
||||
|
||||
note' :: Show s => s -> Test ()
|
||||
note' = note . show
|
||||
|
||||
random :: Random a => Test a
|
||||
random = do
|
||||
rng <- asks rng
|
||||
@ -151,15 +161,78 @@ random = do
|
||||
writeTVar rng rng1
|
||||
pure a
|
||||
|
||||
randomBetween :: Random a => (a,a) -> Test a
|
||||
randomBetween bounds = do
|
||||
random' :: Random a => a -> a -> Test a
|
||||
random' lower upper = do
|
||||
rng <- asks rng
|
||||
liftIO . atomically $ do
|
||||
rng0 <- readTVar rng
|
||||
let (a, rng1) = Random.randomR bounds rng0
|
||||
let (a, rng1) = Random.randomR (lower,upper) rng0
|
||||
writeTVar rng rng1
|
||||
pure a
|
||||
|
||||
int :: Test Int
|
||||
int = random
|
||||
|
||||
char :: Test Char
|
||||
char = random
|
||||
|
||||
double :: Test Double
|
||||
double = random
|
||||
|
||||
word :: Test Word
|
||||
word = random
|
||||
|
||||
word8 :: Test Word8
|
||||
word8 = random
|
||||
|
||||
int' :: Int -> Int -> Test Int
|
||||
int' = random'
|
||||
|
||||
char' :: Char -> Char -> Test Char
|
||||
char' = random'
|
||||
|
||||
double' :: Double -> Double -> Test Double
|
||||
double' = random'
|
||||
|
||||
word' :: Word -> Word -> Test Word
|
||||
word' = random'
|
||||
|
||||
word8' :: Word8 -> Word8 -> Test Word8
|
||||
word8' = random'
|
||||
|
||||
-- | Sample uniformly from the given list of possibilities
|
||||
pick :: [a] -> Test a
|
||||
pick as = let n = length as; ind = picker n as in do
|
||||
i <- int' 0 (n - 1)
|
||||
Just a <- pure (ind i)
|
||||
pure a
|
||||
|
||||
picker :: Int -> [a] -> (Int -> Maybe a)
|
||||
picker _ [] = const Nothing
|
||||
picker _ [a] = \i -> if i == 0 then Just a else Nothing
|
||||
picker size as = go where
|
||||
lsize = size `div` 2
|
||||
rsize = size - lsize
|
||||
(l,r) = splitAt lsize as
|
||||
lpicker = picker lsize l
|
||||
rpicker = picker rsize r
|
||||
go i = if i < lsize then lpicker i else rpicker (i - lsize)
|
||||
|
||||
listOf :: Int -> Test a -> Test [a]
|
||||
listOf = replicateM
|
||||
|
||||
listsOf :: [Int] -> Test a -> Test [[a]]
|
||||
listsOf sizes gen = sizes `forM` \n -> listOf n gen
|
||||
|
||||
pair :: Test a -> Test b -> Test (a,b)
|
||||
pair = liftA2 (,)
|
||||
|
||||
mapOf :: Ord k => Int -> Test k -> Test v -> Test (Map k v)
|
||||
mapOf n k v = Map.fromList <$> listOf n (pair k v)
|
||||
|
||||
mapsOf :: Ord k => [Int] -> Test k -> Test v -> Test [Map k v]
|
||||
mapsOf sizes k v = sizes `forM` \n -> mapOf n k v
|
||||
|
||||
wrap :: Test a -> Test a
|
||||
wrap (Test t) = Test $ do
|
||||
env <- ask
|
||||
|
Loading…
Reference in New Issue
Block a user