fuzzcheck
A simple checker for stress testing monadic code
https://github.com/fpco/fuzzcheck
LTS Haskell 22.33: | 0.1.1 |
Stackage Nightly 2024-09-07: | 0.1.1 |
Latest on Hackage: | 0.1.1 |
fuzzcheck-0.1.1@sha256:f00d2c2eafe4d60e877a1d3d91d6595147ef41682ace3ca8d0dc386f7d70bbe8,1442
Module documentation for 0.1.1
- Test
FuzzCheck
FuzzCheck is a library much like QuickCheck, except that instead of test the properties of pure functions, it tests the behavior of applicative or monadic code.
For example, with QuickCheck you would check a property of a function as follows:
prop_reverse xs = xs == reverse (reverse xs)
>>> quickCheck prop_reverse
This would generate a list of random length and contents, and ensure that the stated property is maintained for each instance.
FuzzCheck is for testing monadic (or applicative) code, which may only be testable in the context of other operations. For example, let’s test some simple FFI code:
prop_bs_ffi = do
mem <- "allocate buffer" ?> pure malloc
n <- "pick a number" ?> return <$> gen (choose (40::Int,100))
"poke" ?> poke <$> arg mem <*> arg n
x <- "peek at memory" ?> peek <$> arg mem
"make sure it matches" ?> (@?=) <$> arg x <*> arg n
"free the buffer" ?> free <$> arg mem
FuzzCheck interface
There are just three special details introduced by FuzzCheck, the ?>
operator, and the arg
, rand
and gen
combinators.
"label" ?> action
This runs a Fuzz
action. If an exception occurs, the label is printed
along with the exception.
let x = "Hello"
"label" ?> f <$> arg x
This executes a monadic function f
, passing it the argument x
. This is
equivalent to using f x
in the surrounding monad, except that if an
exception is generated, the error report looks like this:
f "Hello": <text of actual exception here>
You may also use rand
, which is just a shorter synonym for QuickCheck’s
arbitrary
, for generating a type-appropriate random value automatically:
"label" ?> f <$> rand
Another option is to use gen
, which takes for its argument any combinator
from QuickCheck that generates an appropriately typed Gen
value. For
example:
"label" ?> f <$> gen (choose (1,10))
This tests f
by passing it a randomly chosen integer from the given range.
If an exception occurs, the actual integer that caused the problem is shown:
f 9: <text of actual exception here>
That’s it. To run the test, call fuzzCheck
on the property:
>>> fuzzCheck prop_bs_ffi2
+++ OK, passed 100 tests.
You can use fuzzCheck'
if you want to change the number of tests executed,
or if you want to associate cleanup code with the test after it runs, whether
or not it succeeds.
Simplifying tests
The role of ?>
is to assign a label to each operation (to assist with error
reporting in case of failure), and to execute the Fuzz
action in its
enclosing Monad. A fuzz test may occur within any monad supporting MonadIO
and MonadBaseControl IO
(for the purpose of catching exceptions), which
means that if we’re testing code in IO, we can limit the use of ?>
to only
those cases we expect might fail:
prop_bs_ffi2 = do
mem <- malloc
n <- "pick a number" ?> return <$> gen (choose (40::Int,100))
"poke" ?> poke <$> arg mem <*> arg n
x <- "peek at memory" ?> peek <$> arg mem
x @?= n
free mem
NOTE: Using gen
does not mean that that specific function is invoked 100
times at that point in the monadic block. Instead, the entire block passed to
fuzzCheck
is executed 100 times, with each occurence of gen
producing a
new value at each run.
Integration with Hspec and HUnit
This all integrates quite nicely with Hspec and Hunit. For example, this is from the smoke tests for this library:
hspec $ it "works with an FFI example" $ fuzzCheck $ do
mem <- malloc
n <- "pick a number" ?> return <$> gen (choose (40::Int,100))
"poke" ?> poke <$> arg mem <*> arg n
x <- "peek at memory" ?> peek <$> arg mem
x @?= n
free mem
See also
Although this library was written before I had found
this paper
and Test.QuickCheck.Monadic
, the results are somewhat similar (the paper
uses name
instead of ?>
, while arg
is the same).