Hypothesis: Property-Based Testing for Python

Q: Does Hypothesis work with pytest?

Yes. Tests decorated with @given run as ordinary pytest tests, and Hypothesis adds command-line options such as --hypothesis-show-statistics and --hypothesis-seed.

Hypothesis

Hypothesis is the standard property-based testing library for Python. Instead of writing individual examples, you describe the shape of valid inputs with strategies and the properties your code must satisfy, and Hypothesis generates hundreds of inputs — including the empty strings, huge numbers, and Unicode edge cases that hand-written tests miss. When a property fails, Hypothesis shrinks the input to the smallest failing example and remembers it, so the next run reproduces the bug immediately.

This page is about Hypothesis specifically: its strategies, settings, database, and stateful testing. The general technique — choosing good properties — is covered in Property-Based Testing.

TL;DR

Decorate tests with @given(...) and strategies such as st.integers(), st.text(), and st.lists().
Build domain objects with st.builds, st.from_type, and @st.composite.
Failing examples are shrunk and saved in the example database; pin important ones with @example.
Use settings profiles for fast local runs and deeper CI runs.
Test sequences of operations with stateful testing (RuleBasedStateMachine).
Hypothesis runs inside pytest and unittest with no extra plugins.

Quick Example

Generate valid Money pairs in the same currency and check that addition is commutative and never loses value:

A composite strategy generates exactly the valid inputs the property needs, instead of generating anything and filtering most of it away.

Core Concepts

Strategies

Combinators such as .map(), .filter(), and .flatmap() derive new strategies. Prefer construction (.map, @st.composite) over .filter and assume, which waste generated examples.

Shrinking and the Example Database

When a test fails, Hypothesis searches for a simpler input that still fails — shorter lists, smaller numbers, simpler text — and reports that minimal example. Failing examples are stored in a local example database (the .hypothesis directory) and replayed first on the next run. @example(...) pins important cases permanently so they run on every machine, including CI runners without the database.

Settings and Profiles

settings controls how many examples run (max_examples), time limits (deadline), and health checks. Profiles let local runs stay fast while CI explores more:

Health Checks

Hypothesis warns when a test is too slow, filters out too much generated data, or produces examples that are too large. These checks usually point at a strategy that should be rewritten rather than suppressed.

Stateful Testing

Some bugs appear only after a particular sequence of operations. A rule-based state machine generates random sequences of actions against the real system and a simple model, checking invariants after every step:

When it fails, Hypothesis prints the minimal sequence of steps that reproduces the bug — often three or four operations that would have been very hard to find by hand. Bundle lets rules pass values created by earlier rules, such as IDs of objects created in the system.

Working Effectively

Start with round trips and invariants: serializers, parsers, encoders, and conversions almost always have one.
Model your domain once: put reusable strategies for domain objects in a shared module, mirroring your validation rules.
Keep example-based tests for documented behavior; Hypothesis complements them.
Ghostwriter (hypothesis write module.function) drafts property tests from function signatures as a starting point.
Reproduce CI failures using the printed seed or the falsifying example from the output.

Best Practices

Generate Valid Inputs Directly

Write strategies that produce valid inputs by construction. A test that generates anything and calls assume() on most of it runs slowly and explores little.

Bound Sizes Deliberately

Limit collection sizes and numeric ranges to what is realistic for the property. Unbounded strategies can produce huge inputs that slow tests without finding new bugs.

Commit Regression Examples

When Hypothesis finds a bug, add the minimal example with @example alongside the fix so it stays covered even without the local database. See Security Regression Test Suites for the same idea applied to vulnerabilities.

Run More Examples in CI

Fast local profiles keep the feedback loop short; a CI or nightly profile with many more examples finds the rarer bugs.

Common Mistakes

Filtering Instead of Constructing

Reimplementing the Code in the Property

Disabling Deadlines Everywhere

FAQ

What is Hypothesis?

A Python library for property-based testing. You describe inputs with strategies and state properties that must hold, and Hypothesis generates many inputs, shrinks failures to minimal examples, and remembers them for future runs.

Does Hypothesis work with pytest?

Yes. Tests decorated with @given run as ordinary pytest tests, and Hypothesis adds command-line options such as --hypothesis-show-statistics and --hypothesis-seed.

How do I reproduce a failure from CI?

Use the falsifying example printed in the failure output, add it with @example, or rerun with the reported seed. Saving the example in the test is the most durable fix.

What is stateful testing?

Testing sequences of operations rather than single inputs. A RuleBasedStateMachine defines possible actions and invariants; Hypothesis generates action sequences, compares the system with a model, and shrinks failing sequences to the shortest one.

How many examples should a test run?

The default of 100 suits most local runs. Increase it in CI or nightly profiles for important code, and reduce it for tests with expensive setup. Profiles let you choose per environment.