tasty-rerun

Run tests by filtering the test tree depending on the result of previous test runs

http://github.com/ocharles/tasty-rerun

Version on this page:1.1.8
LTS Haskell 22.39:1.1.19@rev:3
Stackage Nightly 2024-10-31:1.1.19@rev:3
Latest on Hackage:1.1.19@rev:3

See all snapshots tasty-rerun appears in

BSD-3-Clause licensed by Oliver Charles
Maintained by [email protected]
This version can be pinned in stack with:tasty-rerun-1.1.8@sha256:76bfbcc540a83d24c6bc9365b0ac62e168889f4bd08356c8a4cafbd2545baf55,3084

Module documentation for 1.1.8

Used by 1 package in lts-9.21(full list with versions):
git-annex

This ingredient adds the ability to run tests by first filtering the test tree based on the result of a previous test run. For example, you can use this to run only those tests that failed in the last run, or to run only tests that have been added since tests were last ran.

This ingredient is specifically an ingredient *transformer* - given a list of Tasty.Ingredients, rerunningTests adds the ability for all of these ingredients to run against a filtered test tree. This transformer can be applied as follows:

import Test.Tasty
import Test.Tasty.Runners

main :: IO ()
main =
  defaultMainWithIngredients
    [ rerunningTests [ listingTests, consoleTestReporter ] ]
    tests

tests :: TestTree
tests = undefined

This ingredient adds three command line parameters:

--rerun-update
If specified the results of this test run will be saved to the log file at --rerun-log-file. If the ingredient does not execute tests (for example, --list-tests is used) then the log file will not be updated. This option is not enabled by default. This option does not require a value.
--rerun-log-file
The path to the log file to read previous test information from, and where to write new information to (if --rerun-update is specified). This option defaults to .tasty-rerun-log.
--rerun-filter
Which filters to apply to the Tasty.TestTree based on previous test runs. The value of this option is a comma separated list of the following options:

  • failures: Only run tests that failed on the previous run.

  • exceptions: Only run tests that threw an exception on the previous run.

  • new: Only run tests that are new since the previous test run.

  • successful: Only run tests that were successful in the previous run.

Multiple options can be combined and will be taken under disjunction - so --rerun-filter=failures,exceptions will run only tests that failed *or* threw an exception on the last run.

Defaults to all filters, which means all tests will be ran.

Changes

1.1.8

  • Bump upper bound of tasty.

1.1.7

  • Allow base < 4.11.

1.1.6

  • Allow base 4.9 for building with GHC 8.0

1.1.5

  • Supports tasty < 0.12.

1.1.4

  • Supports base <= 4.9, tagged <= 0.9

1.1.3

  • Supports tasty =< 0.11

1.1.2

  • Allow base 4.7 for building with GHC 7.8

1.1.1

  • Update to work with tasty >= 0.8

1.1.0

  • The TestTree is filtered using a custom traversal now, rather than a TreeFold. This gives better guarantees that the TestTree is only reduced and that nodes (such as WithResources) continue to work. The resulting filtered TestTree now has the same shape as the original tree, but filtered tests are transformed into TestGroups with no tests. This is a fairly major change to how the filtering is performed, so this is a new major release, and previous versions are now considered deprecated.

1.0.1

  • Now supports filtering TestTrees that use resources.

1.0.0

  • Initial release. Supports the --rerun-update, --rerun-log-file and --rerun-filter options. Supported filters are new, failures, exceptions and successful.