Proposal to Modify MagicMock.side_effect for Better Function Mocking in Unit Tests

Python Help
1

Over the last few days, I’ve been tackling a problem related to mocking a function that accepts parameters. More specifically, I’ve been trying to test the following logic:

def my_function(df) -> pd.DataFrame:
   return  (
            df.groupby(self.groupby_columns)[
                [column_a, column_b]
            ]
            .apply(my_function_that_call_external_api)
            .reset_index()
        )

def my_function_that_call_external_api(imput_df: pd.DataFrameGroupBy) -> pd.DataFrame:
   df =  preprocess(df)
   predictions =  external_api_call(df)
   return postprocess_predictions(predictions)

For the sake of this discussion, consider external_api_call as a function that calls Prophet, both for fitting and predicting.

To test this functionality, I need to mock the external_api_call. Here’s my attempt:

from unittest.mock import patch
from functools import partial
    def generate_mock_reposnse(self, column_a_modifier: int, column_b_modifie: int,  input_df: pd.DataFrame) -> pd.DataFrame:
        """Generate a dummy external api response with random prediction.
        """
        response = pd.DataFrame(
            {
                "ds":np.full_like(input_df.ds, column_a_modifier, dtype=float),
                "yhat": np.full_like(input_df.ds, column_b_modifie, dtype=float),
            }
        )
        return response

@patch("external_api_call")
    def test_my_function_with_mock(self, api_call_mock):
        df = self.generate_df()
        api_call_mock.side_effect = [partial(generate_mock_reposnse, 
           column_a_modifier = a, 
          column_b_modifie = b)
        for a in a_values
        for b in b_values
             )]
   result = my_function(df)
   self.validate_result(result)

Unfortunately, this code does not work as expected because the partial function is never called. I propose modifying the behavior of MagicMock.side_effect to enable the execution of callable functions.

2

The example code uses a list comprehension to provide a list of partials and stores that in side_effect. The docs for side_effect say:

If you pass in an iterable, it is used to retrieve an iterator which must yield a value on every call. This value can either be an exception instance to be raised, or a value to be returned from the call to the mock (DEFAULT handling is identical to the function case).

So each partial is being returned on each subsequent call.

However, it looks like what you want to do is to preform some additional operations that rely on the value passed in when the mock is called. This could be done using the function capabilities of side_effect.

If you pass in a function it will be called with same arguments as the mock and unless the function returns the DEFAULT singleton the call to the mock will then return whatever the function returns. If the function returns DEFAULT then the mock will return its normal value (from the return_value).

So maybe something like this:

@patch("external_api_call")
def test_my_function_with_mock(self, api_call_mock):
    ab_pairs = ((a, b) for a in a_values for b in b_values)
    def side_effect(imput_df: pd.DataFrameGroupBy) -> pd.DataFrame:
        a, b = next(ab_pairs)
        return generate_mock_reposnse(a, b, imput_df)
    api_call_mock.side_effect = side_effect

    df = self.generate_df()
    result = my_function(df)
    self.validate_result(result)
3

Thank you Patrick.

That would help. Sometimes, when you spend too much time on a given issue, you are not able to find the simplest solution. We can close the topic.