Why my tests fail only during pre-commit ?

Why my tests fail only during pre-commit ?

ยท

2 min read

Symptom

Recently I ran across (what I thought was) strange behaviour.

I use pre-commit for all my git commits, and one of the step is to ensure that all the unit tests pass.

I also have a make target to run just the unit tests.

Each time when I tried commit, the unit test step in the pre-commit would fail - which indicates that the unit tests did not pass. But when I run make unit it would pass. ๐Ÿคท

During the pre-commit, pytest would return with exit code 5 - which indicates no tests ran. So it wasn't that one or more tests failed during pre-commit - but nothing was tested - and due to non-zero exit code, pre-commit prevented the commit (as it should)

Root Cause

It turns out pre-commit runs all the steps only on the modified files. It passes the list of modified files to each command - implicitly. That is, unless you configure it not to. ๐Ÿ˜€

So when I modify a source file (not a test file) - pytest would "ignore" the file - since it would not contain any tests.

Solution

Now that we understand the "why" - solution is obvious.

Patient : Doctor, it hurts when I do this

Doctor : Then don't do that ๐Ÿ˜‚

For the unit test step, ask pre-commit not to pass the file names as an argument via pass_filenames: false

Reference