sampleable Class
This class permits the creation samples of a given type
controlling the size of those values using the gen monad`. It also
helps minimize examples by creating smaller versions of given values.
When testing a proposition like ∀ n : ℕ, prime n → n ≤ 100,
slim_check requires that ℕ have an instance of sampleable and for
prime n to be decidable. slim_check will then use the instance of
sampleable to generate small examples of ℕ and progressively increase
in size. For each example n, prime n is tested. If it is false,
the example will be rejected (not a test success nor a failure) and
slim_check will move on to other examples. If prime n is true, n
≤ 100 will be tested. If it is false, n is a counter-example of ∀
n : ℕ, prime n → n ≤ 100 and the test fails. If n ≤ 100 is true,
the test passes and slim_check moves on to trying more examples.
This is a port of the Haskell QuickCheck library.
Main definitions
sampleableclass
Shrinking
Shrinking happens when slim_check find a counter-example to a
property. It is likely that the example will be more complicated than
necessary so slim_check proceeds to shrink it as much as
possible. Although equally valid, a smaller counter-example is easier
for a user to understand and use.
The sampleable class, beside having the sample function, has a
shrink function so that we can use specialized knowledge while
shrinking a value. It is not responsible for the whole shrinking process
however. It only has to take one step in the shrinking process.
slim_check will repeatedly call shrink until no more steps can
be taken. Because shrink guarantees that the size of the candidates
it produces is strictly smaller than the argument, we know that
slim_check is guaranteed to terminate.
Tags
random testing
References
- https://hackage.haskell.org/package/QuickCheck
- wf : has_sizeof α
- sample : slim_check.gen α
- shrink : Π (x : α), lazy_list {y // sizeof y < sizeof x}
sampleable α provides ways of creating examples of type α,
and given such an example x : α, gives us a way to shrink it
and find simpler examples.
Instances
- slim_check.fin.sampleable'
- slim_check.nat.sampleable
- slim_check.fin.sampleable
- slim_check.pnat.sampleable
- slim_check.int.sampleable
- slim_check.bool.sampleable
- slim_check.prod.sampleable
- slim_check.sum.sampleable
- slim_check.rat.sampleable
- slim_check.char.sampleable
- slim_check.list.sampleable
- slim_check.prop.sampleable
- slim_check.no_shrink.sampleable
- slim_check.string.sampleable
- slim_check.tree.sampleable
- slim_check.nat_le.sampleable
- slim_check.nat_ge.sampleable
- slim_check.nat_gt.sampleable
- slim_check.int_lt.sampleable
- slim_check.int_gt.sampleable
- slim_check.le.sampleable
- slim_check.ge.sampleable
- slim_check.perm.slim_check
- slim_check.perm'.slim_check
nat.shrink' k n creates a list of smaller natural numbers by
successively dividing n by 2 and subtracting the difference from
k. For example, nat.shrink 100 = [50, 75, 88, 94, 97, 99].
Equations
- slim_check.nat.shrink' k n hn ls = dite (n ≤ 1) (λ (h : n ≤ 1), ls.reverse) (λ (h : ¬n ≤ 1), have h₂ : 0 < n, from _, have this : 1 * n / 2 < n, from _, have this : n / 2 < n, from _, let m : ℕ := n / 2 in have h₀ : m ≤ k, from _, have h₃ : 0 < m, from _, have h₁ : k - m < k, from _, slim_check.nat.shrink' k m h₀ (⟨k - m, h₁⟩ :: ls))
nat.shrink n creates a list of smaller natural numbers by
successively dividing by 2 and subtracting the difference from
n. For example, nat.shrink 100 = [50, 75, 88, 94, 97, 99].
Transport a sampleable instance from a type α to a type β using
functions between the two, going in both directions.
Function g is used to define the well-founded order that
shrink is expected to follow.
Equations
- slim_check.nat.sampleable = {wf := nat.has_sizeof, sample := slim_check.gen.sized (λ (sz : ℕ), coe <$> slim_check.gen.choose_any (fin (sz ^ 3).succ) <|> coe <$> slim_check.gen.choose_any (fin sz.succ)), shrink := λ (x : ℕ), lazy_list.of_list (slim_check.nat.shrink x)}
iterate_shrink p x takes a decidable predicate p and a
value x of some sampleable type and recursively shrinks x.
It first calls shrink x to get a list of candidate sample,
finds the first that satisfies p and recursively tries
to shrink that one.
Equations
- slim_check.iterate_shrink p = slim_check.iterate_shrink._proof_1.fix (λ (x : α) (f_rec : Π (y : α), has_well_founded.r y x → option α), trace ("" ++ to_string x ++ (" : " ++ to_string (slim_check.sampleable.shrink x).to_list ++ "")) (λ («_» : unit), pure ()) >>= λ (_x : unit), lazy_list.find (λ (a : {y // sizeof y < sizeof x}), p ↑a) (slim_check.sampleable.shrink x) >>= λ (y : {y // sizeof y < sizeof x}), f_rec ↑y _ <|> some y.val)
Equations
- slim_check.fin.sampleable = slim_check.sampleable.lift ℕ fin.of_nat' subtype.val slim_check.fin.sampleable._proof_1
Equations
- slim_check.fin.sampleable' = slim_check.sampleable.lift ℕ fin.of_nat subtype.val slim_check.fin.sampleable'._proof_1
Equations
- slim_check.pnat.sampleable = slim_check.sampleable.lift ℕ nat.succ_pnat pnat.nat_pred slim_check.pnat.sampleable._proof_1
Equations
- slim_check.int.sampleable = {wf := {sizeof := int.nat_abs}, sample := slim_check.gen.sized (λ (sz : ℕ), let k : ℕ := sz ^ 5 in (λ (n : fin k.succ), ↑(n.val) - int.of_nat (k / 2)) <$> slim_check.gen.choose_any (fin k.succ)), shrink := λ (x : ℤ), lazy_list.of_list ((slim_check.nat.shrink x.nat_abs).bind (λ (_x : {m // has_well_founded.r m x.nat_abs}), slim_check.int.sampleable._match_1 x _x))}
- slim_check.int.sampleable._match_1 x ⟨y, h⟩ = [⟨↑y, h⟩, ⟨-↑y, _⟩]
Equations
- slim_check.bool.sampleable = {wf := bool.has_sizeof, sample := slim_check.gen.choose_any bool >>= λ (x : bool), return x, shrink := λ (_x : bool), lazy_list.nil}
sizeof_lt x y compares the sizes of x and y.
Equations
- slim_check.sizeof_lt x y = (sizeof x < sizeof y)
shrink_fn α is the type of functions that shrink an
argument of type α
Equations
- slim_check.shrink_fn α = Π (x : α), lazy_list {y // slim_check.sizeof_lt y x}
Provided two shrinking functions prod.shrink shrinks a pair (x, y) by
first shrinking x and pairing the results with y and then shrinking
y and pairing the results with x.
All pairs either contain x untouched or y untouched. We rely on
shrinking being repeated for x to get maximally shrunken and then
for y to get shrunken too.
Equations
- slim_check.prod.shrink shr_a shr_b (x₀, x₁) = let xs₀ : lazy_list {y // slim_check.sizeof_lt y (x₀, x₁)} := lazy_list.map (subtype.map (λ (a : α), (a, x₁)) _) (shr_a x₀), xs₁ : lazy_list {y // slim_check.sizeof_lt y (x₀, x₁)} := lazy_list.map (subtype.map (λ (a : β), (x₀, a)) _) (shr_b x₁) in xs₀.append (λ («_» : unit), xs₁)
Equations
- slim_check.prod.sampleable α = {wf := prod.has_sizeof α β slim_check.sampleable.wf, sample := uliftable.up (slim_check.sampleable.sample α) >>= λ (_p : ulift α), slim_check.prod.sampleable._match_2 α _p, shrink := slim_check.prod.shrink slim_check.sampleable.shrink slim_check.sampleable.shrink}
- slim_check.prod.sampleable._match_2 α {down := x} = uliftable.up (slim_check.sampleable.sample β) >>= λ (_p : ulift β), slim_check.prod.sampleable._match_1 α x _p
- slim_check.prod.sampleable._match_1 α x {down := y} = pure (x, y)
shrinking function for sum types
Equations
Equations
Equations
- slim_check.rat.sampleable = slim_check.sampleable.lift (ℤ × ℕ+) (λ (x : ℤ × ℕ+), x.cases_on rat.mk_pnat) (λ (r : ℚ), (r.num, ⟨r.denom, _⟩)) slim_check.rat.sampleable._proof_1
sampleable_char can be specialized into customized sampleable char instances.
The resulting instance has 1 / length chances of making an unrestricted choice of characters
and it otherwise chooses a character from characters with uniform probabilities.
Equations
- slim_check.sampleable_char length characters = {wf := char.has_sizeof, sample := slim_check.gen.choose_nat 0 length _ >>= λ (x : ↥(set.Icc 0 length)), ite (x.val = 0) (slim_check.sampleable.sample ℕ >>= λ (n : ℕ), pure (char.of_nat n)) (slim_check.gen.choose_nat 0 (characters.length - 1) _ >>= λ (i : ↥(set.Icc 0 (characters.length - 1))), pure (characters.mk_iterator.nextn ↑i).curr), shrink := λ (_x : char), lazy_list.nil}
Equations
- slim_check.char.sampleable = slim_check.sampleable_char 3 " 0123abcABC:,;`\\/"
list.shrink_removes shrinks a list by removing chunks of size k in
the middle of the list.
Equations
- slim_check.list.shrink_removes k hk xs n hn = dite (k > n) (λ (hkn : k > n), lazy_list.nil) (λ (hkn : ¬k > n), dite (k = n) (λ (hkn' : k = n), have this : 1 < xs.sizeof, from _, lazy_list.singleton ⟨list.nil α, this⟩) (λ (hkn' : ¬k = n), have h₂ : k < xs.length, from _, slim_check.list.shrink_removes._match_1 k hk xs n hn h₂ (λ (xs₁ xs₂ : list α) (h : (xs₁, xs₂) = list.split_at k xs) (h₄ : xs₁ = list.take k xs) (h₃ : xs₂ = list.drop k xs) (this : sizeof xs₂ < sizeof xs) (h₁ : n - k = xs₂.length) (h₅ : ∀ (a : list α), slim_check.sizeof_lt a xs₂ → slim_check.sizeof_lt (xs₁ ++ a) xs), slim_check.list.shrink_removes k hk xs₂ (n - k) h₁) (list.split_at k xs) _))
- slim_check.list.shrink_removes._match_1 k hk xs n hn h₂ _f_1 (xs₁, xs₂) h = have h₄ : xs₁ = list.take k xs, from _, have h₃ : xs₂ = list.drop k xs, from _, have this : sizeof xs₂ < sizeof xs, from _, have h₁ : n - k = xs₂.length, from _, have h₅ : ∀ (a : list α), slim_check.sizeof_lt a xs₂ → slim_check.sizeof_lt (xs₁ ++ a) xs, from _, lazy_list.cons ⟨xs₂, this⟩ (λ («_» : unit), subtype.map (append xs₁) h₅ <$> _f_1 xs₁ xs₂ h h₄ h₃ this h₁ h₅)
list.shrink_one xs shrinks list xs by shrinking only one item in
the list.
Equations
- slim_check.list.shrink_one shr (x :: xs) = (subtype.map (λ (x' : α), x' :: xs) _ <$> shr x).append (λ («_» : unit), subtype.map ((λ (_x : α) (_y : list α), _x :: _y) x) _ <$> slim_check.list.shrink_one shr xs)
- slim_check.list.shrink_one shr list.nil = lazy_list.nil
list.shrink_with shrink_f xs shrinks xs by first
considering xs with chunks removed in the middle (starting with
chunks of size xs.length and halving down to 1) and then
shrinks only one element of the list.
This strategy is taken directly from Haskell's QuickCheck
Equations
- slim_check.list.shrink_with shr xs = let n : ℕ := xs.length in ((lazy_list.cons n (λ («_» : unit), lazy_list.map subtype.val (slim_check.sampleable.shrink n).reverse)).bind (λ (k : ℕ), dite (0 < k) (λ (hk : 0 < k), slim_check.list.shrink_removes k hk xs n rfl) (λ (hk : ¬0 < k), lazy_list.nil))).append (λ («_» : unit), slim_check.list.shrink_one shr xs)
Equations
- slim_check.prop.sampleable = {wf := default_has_sizeof Prop, sample := slim_check.gen.choose_any bool >>= λ (x : bool), return ↑x, shrink := λ (_x : Prop), lazy_list.nil}
no_shrink is a type annotation to signal that
a certain type is not to be shrunk. It can be useful in
combination with other types: e.g. xs : list (no_shrink ℤ)
will result in the list being cut down but individual
integers being kept as is.
Equations
Equations
- slim_check.inhabited = {default := default α _inst_1}
Introduction of the no_shrink type.
Equations
Selector of the no_shrink type.
Equations
Equations
- slim_check.string.sampleable = {wf := slim_check.sampleable.wf (slim_check.sampleable.lift (list char) list.as_string string.to_list slim_check.string.sampleable._proof_1), sample := (slim_check.sampleable.sample char).list_of >>= λ (x : list char), pure x.as_string, shrink := slim_check.sampleable.shrink (slim_check.sampleable.lift (list char) list.as_string string.to_list slim_check.string.sampleable._proof_1)}
implementation of sampleable (tree α)
Equations
- slim_check.tree.sample sample n = ite (n > 0) (tree.node <$> sample <*> slim_check.tree.sample sample (n / 2) <*> slim_check.tree.sample sample (n / 2)) (pure tree.nil)
rec_shrink x f_rec takes the recursive call f_rec introduced
by well_founded.fix and turns it into a shrinking function whose
result is adequate to use in a recursive call.
Equations
- slim_check.rec_shrink t sh ⟨t', ht'⟩ = (λ (t'' : {y // slim_check.sizeof_lt y t'}), ⟨⟨t''.val, _⟩, _⟩) <$> sh t' ht'
tree.shrink_with shrink_f t shrinks xs by using the empty tree,
each subtrees, and by shrinking the subtree to recombine them.
This strategy is taken directly from Haskell's QuickCheck
Equations
- slim_check.tree.shrink_with shrink_a = slim_check.tree.shrink_with._proof_1.fix (λ (t : tree α), slim_check.tree.shrink_with._match_1 shrink_a t t)
- slim_check.tree.shrink_with._match_1 shrink_a t (tree.node x t₀ t₁) = λ (f_rec : Π (y : tree α), sizeof_measure (tree α) y (tree.node x t₀ t₁) → lazy_list {y_1 // slim_check.sizeof_lt y_1 y}), let shrink_tree : slim_check.shrink_fn {t' // slim_check.sizeof_lt t' (tree.node x t₀ t₁)} := λ (t' : {t' // slim_check.sizeof_lt t' (tree.node x t₀ t₁)}), slim_check.rec_shrink (tree.node x t₀ t₁) f_rec t' in have h₂ : slim_check.sizeof_lt tree.nil (tree.node x t₀ t₁), from _, have h₀ : slim_check.sizeof_lt t₀ (tree.node x t₀ t₁), from _, have h₁ : slim_check.sizeof_lt t₁ (tree.node x t₀ t₁), from _, (lazy_list.of_list [lazy_list.of_list [⟨tree.nil α, h₂⟩, ⟨t₀, h₀⟩, ⟨t₁, h₁⟩]]).join.append (λ («_» : unit), lazy_list.map (λ (_x : {y // slim_check.sizeof_lt y (x, ⟨t₀, h₀⟩, ⟨t₁, h₁⟩)}), slim_check.tree.shrink_with._match_2 x t₀ t₁ h₀ h₁ _x) (slim_check.prod.shrink shrink_a (slim_check.prod.shrink shrink_tree shrink_tree) (x, ⟨t₀, h₀⟩, ⟨t₁, h₁⟩)))
- slim_check.tree.shrink_with._match_1 shrink_a t tree.nil = λ (f_rec : Π (y : tree α), sizeof_measure (tree α) y tree.nil → lazy_list {y_1 // slim_check.sizeof_lt y_1 y}), lazy_list.nil
- slim_check.tree.shrink_with._match_2 x t₀ t₁ h₀ h₁ ⟨(y, ⟨t'₀, _x⟩, ⟨t'₁, _x_1⟩), hy⟩ = ⟨tree.node y t'₀ t'₁, _⟩
Equations
- slim_check.tree.sampleable = {wf := tree.has_sizeof_inst α slim_check.sampleable.wf, sample := slim_check.gen.sized (slim_check.tree.sample (slim_check.sampleable.sample α)), shrink := slim_check.tree.shrink_with slim_check.sampleable.shrink}
Subtype instances
The following instances are meant to improve the testing of properties of the form
∀ i j, i ≤ j, ...
The naive way to test them is to choose two numbers i and j and check that
the proper ordering is satisfied. Instead, the following instances make it
so that j will be chosen with considerations to the required ordering
constraints. The benefit is that we will not have to discard any choice
of j.
Equations
- slim_check.nat_le.sampleable = {wf := subtype.has_sizeof (λ (x : ℕ), x ≤ y), sample := slim_check.gen.choose_nat 0 y slim_check.nat_le.sampleable._proof_1 >>= λ (_p : ↥(set.Icc 0 y)), slim_check.nat_le.sampleable._match_1 _p, shrink := λ (_x : {x // x ≤ y}), lazy_list.nil}
- slim_check.nat_le.sampleable._match_1 ⟨x, h⟩ = pure ⟨x, _⟩
Equations
- slim_check.nat_ge.sampleable = {wf := subtype.has_sizeof (λ (y : ℕ), x ≤ y), sample := slim_check.sampleable.sample ℕ >>= λ (_p : ℕ), slim_check.nat_ge.sampleable._match_1 _p, shrink := λ (_x : {y // x ≤ y}), lazy_list.nil}
- slim_check.nat_ge.sampleable._match_1 y = pure ⟨x + y, _⟩
Equations
- slim_check.nat_gt.sampleable = {wf := subtype.has_sizeof (λ (y : ℕ), x < y), sample := slim_check.sampleable.sample ℕ >>= λ (_p : ℕ), slim_check.nat_gt.sampleable._match_1 _p, shrink := λ (_x : {y // x < y}), lazy_list.nil}
- slim_check.nat_gt.sampleable._match_1 y = pure ⟨x + y + 1, _⟩
Equations
- slim_check.int_lt.sampleable = {wf := subtype.has_sizeof (λ (x : ℤ), x < y), sample := slim_check.sampleable.sample ℕ >>= λ (x : ℕ), pure ⟨y - (↑x + 1), _⟩, shrink := λ (_x : {x // x < y}), lazy_list.nil}
Equations
- slim_check.le.sampleable = {wf := subtype.has_sizeof (λ (x : α), x ≤ y), sample := slim_check.sampleable.sample α >>= λ (x : α), pure ⟨y - abs x, _⟩, shrink := λ (_x : {x // x ≤ y}), lazy_list.nil}
Equations
- slim_check.ge.sampleable = {wf := subtype.has_sizeof (λ (y : α), x ≤ y), sample := slim_check.sampleable.sample α >>= λ (y : α), pure ⟨x + abs y, _⟩, shrink := λ (_x : {y // x ≤ y}), lazy_list.nil}
Equations
- slim_check.perm.slim_check = {wf := subtype.has_sizeof (λ (ys : list α), xs ~ ys), sample := slim_check.gen.permutation_of xs, shrink := λ (_x : {ys // xs ~ ys}), lazy_list.nil}
Equations
- slim_check.perm'.slim_check = {wf := subtype.has_sizeof (λ (ys : list α), ys ~ xs), sample := subtype.map id list.perm.symm <$> slim_check.gen.permutation_of xs, shrink := λ (_x : {ys // ys ~ xs}), lazy_list.nil}
Print (at most) 10 samples of a given type to stdout for debugging.
Equations
- slim_check.print_samples t = io.run_rand (uliftable.down (mmap ((slim_check.sampleable.sample t).run ∘ ulift.up) (list.range 10) >>= λ (xs : list t), pure {down := list.map repr xs})) >>= λ (xs : list string), mmap' io.put_str_ln xs
#sample my_type, where my_type has an instance of sampleable, prints ten random
values of type my_type of using an increasing size parameter.
#sample nat
-- prints
-- 0
-- 0
-- 2
-- 24
-- 64
-- 76
-- 5
-- 132
-- 8
-- 449
-- or some other sequence of numbers
#sample list int
-- prints
-- []
-- [1, 1]
-- [-7, 9, -6]
-- [36]
-- [-500, 105, 260]
-- [-290]
-- [17, 156]
-- [-2364, -7599, 661, -2411, -3576, 5517, -3823, -968]
-- [-643]
-- [11892, 16329, -15095, -15461]
-- or whatever