Include prefix 'called_' in list of forbidden method prefixes for mock objects in unsafe mode

Related to

I’ve noticed a lot of wrong mock usages in the form of

assert mock_object.called_once_with()

The method called_once_with returns an instance of mock.Mock which evaluates to true and therefore passes the assertion.

It is not impossible but quite difficult to detect with static analysis. Basically the same arguments from previous tickets apply here.

I already have a patch and would like to add this as a ticket to github.

See example from stdlib: cpython/ at main · python/cpython · GitHub

Why is it wrong?

The method doesn’t exist and therefore the return value is an instance of Mock. An instance of mock evaluates to true, which makes this a tautology test and thus gives a false sense of safety.

See this short snippet of the passing assertion:

In [1]: from unittest import mock

In [2]: m = mock.Mock()

In [3]: m.method(param=1)
Out[3]: <Mock name='mock.method()' id='4353968160'>

In [4]: m.method.called_once_with(arg=2)
Out[4]: <Mock name='mock.method.called_once_with()' id='4387616944'>

In [5]: assert m.method.called_once_with(arg=2)

When in fact the test should fail:

In [14]: m.method.assert_called_once_with(arg=2)
AssertionError                            Traceback (most recent call last)
Cell In [14], line 1
----> 1 m.method.assert_called_once_with(arg=2)

File /opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/, in NonCallableMock.assert_called_once_with(self, *args, **kwargs)
    936     msg = ("Expected '%s' to be called once. Called %s times.%s"
    937            % (self._mock_name or 'mock',
    938               self.call_count,
    939               self._calls_repr()))
    940     raise AssertionError(msg)
--> 941 return self.assert_called_with(*args, **kwargs)

File /opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/unittest/, in NonCallableMock.assert_called_with(self, *args, **kwargs)
    927 if actual != expected:
    928     cause = expected if isinstance(expected, Exception) else None
--> 929     raise AssertionError(_error_message()) from cause

AssertionError: expected call not found.
Expected: method(arg=2)
Actual: method(param=1)

The IDLE test, being fixed by gh_100647, is the only one in the stdlib. So I am not convinced that this is needed. If you have data that this mistake is near as common as checking for ‘assert’ misspelled as ‘assret’, it would a plausible addition.

Yes, there was only one occurrence. Compared to 0 for each of ‘assert’, ‘assret’, ‘asert’, ‘aseert’, ‘assrt’ in the stdlib. In my opinion, it’s much more likely to make this mistake than one of the typos, because it looks correct and there could be code like

assert mock_f.called
assert mock_f.called_with(param=foo)

A quick search on github shows some results and in our private (multi million line) code base, I found more than 500 across all repositories, with more than half of it not passing after fixing the call from assert mock_f.called_once_with to mock_f.assert_called_once_with().

The mock module already does a good job to protect against trivial typos, but I think it could even do a better job protecting against this error class. The functionality already exists and just needs to be extended. As I’ve written in the initial post, I already have a patch, which correctly detected the wrong assertion in idle.

Go ahead and submit an issue with the data you added and a PR. On the issue, ping me and the people on the previous issue and the unittest people.

Instead of the prefix called_, why not all the known methods and attributes with prefix assert_ removed? That allows any real code using that prefix (eg called_by_first_name in some hypothetical international people-management library) to not fail the test.

1 Like

This was also suggested on the the PR: gh-100690: Prevent prefix "called_" for attributes of safe mock objects by cklein · Pull Request #100691 · python/cpython · GitHub

In that case, I think it would make more sense to have a denylist of names rather than using prefixes. I just updated the pull request.

Note that according to the error message one should be able to provide a spec for their mock object, but this seems to be broken, as commented by myself on the PR.

Now that Mock spec not respected for attributes prefixed with assert · Issue #100739 · python/cpython · GitHub was fixed, I added a test that shows that by using a spec, one can still have mock objects with attribute names like called_once.
The implementation was also moved from checking the prefix called_ to having a denylist of attribute names that is dynamically generated from the attribute names of NonCallableMock that start with assert_ to also catch has_calls like reported in The "has_calls" is used in place of "assert_has_calls" in a few places · Issue #20453 · apache/airflow · GitHub