tasty-discover
Test discovery for the tasty framework.
https://github.com/haskell-works/tasty-discover
LTS Haskell 22.33: | 5.0.0@rev:1 |
Stackage Nightly 2024-09-07: | 5.0.0@rev:1 |
Latest on Hackage: | 5.0.0@rev:1 |
tasty-discover-5.0.0@sha256:2799d2aa77c124e3f842abed9db7e8df260fe3d786549c293a322889c05484a8,6074
Module documentation for 5.0.0
- Test
tasty-discover
- Getting Started
- Write Tests
- Customise Discovery
- Example Project
- Change Log
- Deprecation Policy
- Contributing
- FAQ
- Maintenance
- Acknowledgements
Haskell auto-magic test discovery and runner for the tasty test framework.
Prefix your test case names and tasty-discover
will discover, collect and run
them. All popular Haskell test libraries are covered. Configure once then just
write your tests. Remember to add your test modules to your Cabal/Hpack
files. Tasty ingredients are included along with various configuration options
for different use cases.
See below for full documentation and examples.
Getting Started
There are 4 simple steps:
- Create a test driver file in the test directory
- Mark the driver file as the
main-is
in the test suite - Mark tests with the correct prefixes
- Customise test discovery as needed
Check out the example project to get moving quickly.
Create Test Driver File
You can name this file anything you want but it must contain the correct
preprocessor definition for tasty-discover
to run and to detect the
configuration. It should be at the top level of the test directory.
For example (in test/Driver.hs
):
{-# OPTIONS_GHC -F -pgmF tasty-discover #-}
Configure Cabal or Hpack Test Suite
In order for Cabal/Stack to know where the tests are, you’ll need to configure
the main-is
option of your test-suite to point to the driver file. In the
following example, the test driver file is called Driver.hs
:
test-suite test
main-is: Driver.hs
hs-source-dirs: test
build-depends: base
If you use hpack, that might look like:
tests:
test:
main: "Driver.hs"
source-dirs: "test"
dependencies:
- "base"
To ensure that tasty-discover
is available even without installation, add this
to the test suite in your cabal file:
build-tool-depends:
tasty-discover:tasty-discover
See hpack
documentation for stack
equivalent.
Write Tests
Create test modules and prefix the test function name with an identifier that corresponds to the testing library you wish to run the test with:
- prop_: QuickCheck properties.
- scprop_: SmallCheck properties.
- hprop_: Hedgehog properties.
- unit_: HUnit test cases.
- spec_: Hspec specifications.
- test_: Tasty TestTrees.
- tasty_: Custom tests
Here is an example test module with a bunch of different tests:
{-# LANGUAGE ScopedTypeVariables #-}
module ExampleTest where
import Data.List
import Test.Tasty
import Test.Tasty.Discover
import Test.Tasty.HUnit
import Test.Tasty.Hspec
import Test.Tasty.QuickCheck
-- HUnit test case
unit_listCompare :: IO ()
unit_listCompare = [1, 2, 3] `compare` [1,2] @?= GT
-- QuickCheck property
prop_additionCommutative :: Int -> Int -> Bool
prop_additionCommutative a b = a + b == b + a
-- SmallCheck property
scprop_sortReverse :: [Int] -> Bool
scprop_sortReverse list = sort list == sort (reverse list)
-- Hspec specification
spec_prelude :: Spec
spec_prelude = describe "Prelude.head" $ do
it "returns the first element of a list" $ do
head [23 ..] `shouldBe` (23 :: Int)
-- Custom test
--
-- Write a test for anything with a Tasty instance
--
-- In order to use this feature, you must add tasty-discover as a library dependency
-- to your test component in the cabal file.
--
-- The instance defined should not be an orphaned instance. A future version of
-- tasty-discover may choose to define orphaned instances for popular test libraries.
import Test.Tasty (testCase)
import Test.Tasty.Discover (TestCase(..), descriptionOf)
data CustomTest = CustomTest String Assertion
instance Tasty CustomTest where
tasty info (CustomTest prefix act) =
pure $ testCase (prefix ++ descriptionOf info) act
tasty_myTest :: CustomTest
tasty_myTest = CustomTest "Custom: " $ pure ()
-- Tasty TestTree
test_multiplication :: [TestTree]
test_multiplication = [testProperty "One is identity" $ \(a :: Int) -> a * 1 == a]
-- Tasty IO TestTree
test_generateTree :: IO TestTree
test_generateTree = do
input <- pure "Some input"
pure $ testCase input $ pure ()
-- Tasty IO [TestTree]
test_generateTrees :: IO [TestTree]
test_generateTrees = do
inputs <- pure ["First input", "Second input"]
pure $ map (\s -> testCase s $ pure ()) inputs
Customise Discovery
You configure tasty-discover
by passing options to the test driver file.
No Arguments
Example: {-# OPTIONS_GHC -F -pgmF tasty-discover -optF --debug #-}
- –debug: Output the contents of the generated module while testing.
- –tree-display: Display the test output results hierarchically.
With Arguments
Example: {-# OPTIONS_GHC -F -pgmF tasty-discover -optF --modules="*CustomTest.hs" #-}
- –modules: Which test modules to discover (with glob pattern).
- –search-dir: Where to look for tests. This is a directory relative to the location of the source file. By default, this is the directory of the source file.”
- –ignores: Which test modules to ignore (with glob pattern).
- –generated-module: The name of the generated test module.
- –ingredient: Tasty ingredients to add to your test runner.
- –inplace: Has the generated code written to the source file.
It is also possible to override tasty test options with -optF
:
{-# OPTIONS_GHC -F -pgmF tasty-discover -optF --hide-successes #-}
Example Project
See the testing for this package for a fully configured example.
Change Log
Please see the CHANGELOG.md for the latest changes.
We try to keep tagged releases in our release process, if you care about that.
Deprecation Policy
If a breaking change is implemented, you’ll see a major version increase, an entry in the change log and a compile-time error with a deprecation warning and clear instructions on how to upgrade. Please do complain if we’re doing this too much.
Contributing
All contributions welcome! The continuous integration suite is pretty comprehensive, so just get hacking and add a test case - there are plenty of examples, so this should be simple - and I’ll get to review your change ASAP.
Frequently Asked Questions
Deleting Tests Breaks The Test Run
This is a known limitation and has been reported. No fix is planned unless you have time.
Please see #145 for more information.
Maintenance
If you’re interested in helping maintain this package, please let @newhoggy know!
It doesn’t take much time (max ~3 hours a month) and all we need to do is:
- Triage issues that are raised.
- Review pull requests from contributors.
- Fix bugs when present.
- Make releases.
- Manage bounds issues on Stackage.
You can create an issue or drop him a line at lukewm AT riseup DOT NET.
Acknowledgements
Thanks to hspec-discover and tasty-auto for making this possible.
A huge thanks to the growing list of contributors.
Changes
Change Log
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
5.0.0 [2022-07-08]
- Fix tasty-hedgehog
testProperty
deprecation warning
4.2.4 [2022-05-22]
- Support for custom test libraries
- Version module
- Deduplicate imports in generated code
- Rename library directory to src
- Move existing library modules to Test.Discover.Internal
4.2.3 [2022-05-21]
- Added
--search-dir DIR
option - Adds an
--in-place
flag to write the generated driver to the source file.
4.2.1 [2018-06-06]
Changed
- Migrated source code hosting to https://git.coop/lwm/tasty-discover.
4.2.0 [2018-03-03]
Fixed
- Actually support
scprop_
prefixed SmallCheck test cases (see issue #140).
4.1.5 [2018-02-26]
Fixed
- Fixed multi-byte string issue (see pull request #138).
4.1.4 [2018-02-25]
Added
- Windows OS continuous integration build (see pull request #136).
Fixed
- Test failure related to path handling on Windows OS (see pull request #136).
- Resolved upstream tasty-hedgehog nightly blocking issue (see issue #131).
4.1.3 [2018-01-01]
Fixed
- Re-enable on Stackage due to tasty/tasty-hedgehog failure (see issue #132).
4.1.2 [2017-12-19]
Fixed
- Escaping issues for the Windows platform (see issue #124).
4.1.1 [2017-09-26]
Fixed
- Incorrect test case doing bad comparison (see issue #123).
4.1.0 [2017-09-26]
Fixed
- Find tests recursively in test directory. (see pull request #122).
Added
- Add ability to override tasty arguments (see pull request #120).
4.0.0 [2017-09-01]
Changed
- Deprecated
--[no-]module-suffix
for--modules
(see pull request #117). - Deprecated
--ignore-module
for--ignores
(see pull request #117).
Added
tasty-hedgehog
is now a supported test library.
Removed
case_
prefixes have been removed.
3.0.2 [2017-06-05]
Fixed
- Make upper bounds for dependencies looser.
- Fix typo in README.md option documentation.
Remove
- Remove TOC, the hyperlinks weren’t working on Hackage.
3.0.1 [2017-06-04]
Fixed
- Fixed CHANGELOG.md rendering for Hackage (see pull request #106).
Added
- Add missing –tree-display documentation note (see pull request #107).
3.0.0 [2017-06-03]
Added
- Add –tree-display configuration option (see pull request #103).
Changed
- Deprecate
case_
in favour ofunit_
for HUnit test cases (see pull request #97).
Fixed
- Correctly handle sub-directories when using –no-module-suffix (see pull request #102).
2.0.3 [2017-04-13]
Fixed
- Make the Cabal description more clear for Hackage.
2.0.2 [2017-04-13]
Added
- README.md and CHANGELOG.md included for Hackage (see pull request #96).
- Re-add stylish-haskell automated checking (see pull request #88).
2.0.1 [2017-03-18]
Fixed
- Fix flaky test comparison (see pull request #86).
Removed
- Remove the Test.Tasty.Type module (see pull request #83).
2.0.0 [2017-03-15]
Added
- Add new hpack format.
- Add generator style test discovery from tasty-auto.
- Add new configuration options: debug, ingredients and module name.
- Add unit tests for all functionality.
Fixed
- Re-license to MIT.
Removed
- RTD documentation.
- TemplateHaskell dependency
- Example project and integration test project.
Changed
- Move all tests into test folder.
1.1.0 [2017-01-19]
Added
- Add –ignore-module configuration option.
1.0.1 [2017-11-13]
Added
- Add Cabal and Documentation testing on Travis CI.
Fixed
- Include missing extra-source-files.
- Slim down LICENSE.md and mark as GPL-3 in Cabal file.
1.0.0 [2016-11-04]
Added
- Add documentation on RTD.
- Release on Hackage and Stackage.
0.0.3 [2016-09-20]
Added
- –no-module-suffix configuration option.
0.0.2 [2016-02-20]
Added
- –module-suffix configuration option.
0.0.1 [2016-02-13]
- tasty-discover initial release.