Pytest wrapping output written to stdout causing assertions to fail

Python Help
1

I have a function that prints a directory table:

table = """
| Name                                           | Date modified             | Type   | Size     | Compressed Size |
|------------------------------------------------|---------------------------|--------|----------|-----------------|
| source-main                                    | 2024-04-10T20:10:57+00:00 | Folder | 0B       | 0B              |
| source-main/.github                            | 2024-04-10T20:10:57+00:00 | Folder | 0B       | 0B              |
| source-main/.github/workflows                  | 2024-04-10T20:10:57+00:00 | Folder | 0B       | 0B              |
| source-main/.github/workflows/docs.yml         | 2024-04-10T20:10:57+00:00 | File   | 1.2KiB   | 1.2KiB          |
| source-main/.github/workflows/release.yml      | 2024-04-10T20:10:57+00:00 | File   | 1.2KiB   | 1.2KiB          |
| source-main/.github/workflows/test.yml         | 2024-04-10T20:10:57+00:00 | File   | 1.9KiB   | 1.9KiB          |
"""

def test_print_table(capsys):
        print_table("path")
        assert capsys.readouterr().out.strip() == table.strip()

This test fails, because capsys seems to truncate the output?

E             - | source-main/.github/workflows/docs.yml         | 2024-04-10T20:10:57+00:00 | File   | 1.2KiB   | 1.2KiB          |
E             ?                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                    ^^^^^^^^
E             + | source-main/.… | 2024-04-10T20:10:… | File   | 1.2KiB   | 1.2KiB          |
E             ?                   ^                    ^
E             - | source-main/.github/workflows/release.yml      | 2024-04-10T20:10:57+00:00 | File   | 1.2KiB   | 1.2KiB          |
E             ?                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                    ^^^^^^^^
E             + | source-main/.… | 2024-04-10T20:10:… | File   | 1.2KiB   | 1.2KiB          |
E             ?                   ^                    ^
E             - | source-main/.github/workflows/test.yml         | 2024-04-10T20:10:57+00:00 | File   | 1.9KiB   | 1.9KiB          |
E             ?                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                    ^^^^^^^^
E             + | source-main/.… | 2024-04-10T20:10:… | File   | 1.9KiB   | 1.9KiB          |

My issue seems to be the same as python - Make pytest 'capsys' fixture treat stdout the same regardless of whether or not the -s option is used - Stack Overflow but the solutuion doesn’t apply because im not using argparse, but instead using rich.table — Rich 13.6.0 documentation

The -s option as noted in the SO works on my local windows machine but fails in the CI with the same wrapping assertion fail.

2

Hardcoding time-stamps into a test is seldom a good idea. This was bound to be a flakey test sooner or later.

Try splitting the string into multiple lines, and testing each line individually. The error message will be more helpful then too.

3

After a bit of testing, I’ve managed to get a reproducible code snippet:

from pytest import CaptureFixture

from rich.table import Table
from rich import print, box
from datetime import datetime

expected = """
| Released                   | Title                            | Box Office   | Genre           | Director    |
|----------------------------|----------------------------------|--------------|-----------------|-------------|
| 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |
"""

def test_stdout(capsys: CaptureFixture[str]) -> None:
    table = Table(title="", box=box.MARKDOWN)

    table.add_column("Released", no_wrap=True)
    table.add_column("Title", no_wrap=True)
    table.add_column("Box Office", no_wrap=True)
    table.add_column("Genre", no_wrap=True)
    table.add_column("Director", no_wrap=True)

    table.add_row(
        datetime.max.isoformat(),
        "Star Wars: The Rise of Skywalker",
        "$952,110,690",
        "Sci-Fi, Fantasy",
        "J.J. Abrams",
    )
    print(table)
    # print(table, file=open("stdout.txt", "w"))
    assert capsys.readouterr().out.strip() == expected.strip()
    # open("capsys.txt", "w").write(capsys.readouterr().out)
Error 
❯ pytest -vvvv
=================================================================================== test session starts ===================================================================================
platform win32 -- Python 3.12.3, pytest-8.2.0, pluggy-1.5.0 -- C:\Users\monarch\Documents\GitHub\abcdef\.venv\Scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\Users\monarch\Documents\GitHub\abcdef
configfile: pyproject.toml
collected 1 item

tests/test_stdout.py::test_stdout FAILED                                                                                                                                             [100%]

======================================================================================== FAILURES =========================================================================================
_______________________________________________________________________________________ test_stdout _______________________________________________________________________________________

capsys = <_pytest.capture.CaptureFixture object at 0x00000266B3386690>

    def test_stdout(capsys: CaptureFixture[str]) -> None:
        table = Table(title="", box=box.MARKDOWN)

        table.add_column("Released", no_wrap=True)
        table.add_column("Title", no_wrap=True)
        table.add_column("Box Office", no_wrap=True)
        table.add_column("Genre", no_wrap=True)
        table.add_column("Director", no_wrap=True)

        table.add_row(
            datetime.max.isoformat(),
            "Star Wars: The Rise of Skywalker",
            "$952,110,690",
            "Sci-Fi, Fantasy",
            "J.J. Abrams",
        )
        print(table)
        # print(table, file=open("stdout.txt", "w"))
>       assert capsys.readouterr().out.strip() == expected.strip()
E       AssertionError: assert '| Released            | Title                      | Box … | Genre     | Dir… |\n|---------------------|----------------------------|-------|-----------|------|\n| 9999-12-31T23:59:5… | Star Wars: The Rise of Sk… | $952… | Sci-Fi, … | J.J… |' == '| Released                   | Title                            | Box Office   | Genre
| Director    |\n|----------------------------|----------------------------------|--------------|-----------------|-------------|\n| 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |'
E
E         - | Released                   | Title                            | Box Office   | Genre           | Director    |
E         ?                       -------                             ------      ^^^^^^^^             ------     ^^^^^^^^
E         + | Released            | Title                      | Box … | Genre     | Dir… |
E         ?                                                          ^                  ^
E         - |----------------------------|----------------------------------|--------------|-----------------|-------------|
E         ?  -------                                                  ------ -------                   ------ -------
E         + |---------------------|----------------------------|-------|-----------|------|
E         - | 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |
E         ?                     ^^^^^^^^                            ^^^^^^^       ^^^^^^^^           ^^^^^^^      ^^^^^^^^
E         + | 9999-12-31T23:59:5… | Star Wars: The Rise of Sk… | $952… | Sci-Fi, … | J.J… |
E         ?                     ^                            ^       ^           ^      ^

tests\test_stdout.py:31: AssertionError
================================================================================= short test summary info =================================================================================
FAILED tests/test_stdout.py::test_stdout - AssertionError: assert '| Released            | Title                      | Box … | Genre     | Dir… |\n|---------------------|----------------------------|-------|-----------|------|\n| 9999-12-31T23:59:5… | Star Wars: The Rise of Sk… | $952… | Sci-Fi, … | J.J… |' == '| Released                   | Title
  | Box Office   | Genre           | Director    |\n|----------------------------|----------------------------------|--------------|-----------------|-------------|\n| 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |'

  - | Released                   | Title                            | Box Office   | Genre           | Director    |
  ?                       -------                             ------      ^^^^^^^^             ------     ^^^^^^^^
  + | Released            | Title                      | Box … | Genre     | Dir… |
  ?                                                          ^                  ^
  - |----------------------------|----------------------------------|--------------|-----------------|-------------|
  ?  -------                                                  ------ -------                   ------ -------
  + |---------------------|----------------------------|-------|-----------|------|
  - | 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |
  ?                     ^^^^^^^^                            ^^^^^^^       ^^^^^^^^           ^^^^^^^      ^^^^^^^^
  + | 9999-12-31T23:59:5… | Star Wars: The Rise of Sk… | $952… | Sci-Fi, … | J.J… |
  ?                     ^                            ^       ^           ^      ^
==================================================================================== 1 failed in 0.18s ====================================================================================

What actually gets captured by capsys looks like this:

| Released            | Title                      | Box ďż˝ | Genre     | Dirďż˝ |
|---------------------|----------------------------|-------|-----------|------|
| 9999-12-31T23:59:5ďż˝ | Star Wars: The Rise of Skďż˝ | $952ďż˝ | Sci-Fi, ďż˝ | J.Jďż˝ |
4

Not capsys (the Pytest fixture), but Rich (the actual output seen by Pytest). You could have found this out by trying the code manually, outside the testing framework.

Your test expects the table output by Rich to show every column as wide as necessary to accomodate the text. But Rich doesn’t work that way, at least as you have it configured: it’s aware of the terminal’s width, and won’t make the table wider than that - and will abbreviate columns to make them fit. The �s you’re seeing are replacement characters output by your terminal because it either doesn’t properly understand the encoded output, or (more likely) doesn’t have a glyph for the character. On my terminal, I see ellipsis characters: …. If you actually expand the width of the terminal window, you can make the output appear properly:

>>> rich.print(table) # with the terminal at the default width
                                                                                
| Released             | Title                      | Box … | Genre     | Dir… |
|----------------------|----------------------------|-------|-----------|------|
| 9999-12-31T23:59:59… | Star Wars: The Rise of Sk… | $952… | Sci-Fi, … | J.J… |
                                                                                
>>> rich.print(table) # after making it wider
                                                                                                                
| Released                   | Title                            | Box Office   | Genre           | Director    |
|----------------------------|----------------------------------|--------------|-----------------|-------------|
| 9999-12-31T23:59:59.999999 | Star Wars: The Rise of Skywalker | $952,110,690 | Sci-Fi, Fantasy | J.J. Abrams |

But that sort of thing is not reliable for testing. Rich isn’t really designed for making reproducible, testable output; it’s designed for making nice terminal UIs.

There is a way, however. You can create a Console that has an explicitly specified width that’s large enough to fit your data, and use its print rather than the default Rich print:

>>> from rich.console import Console
>>> c = Console(width=200)
>>> c.print(table)

There might be other ways that I didn’t figure out :wink:

That said, instead of trying to verify the exact contents of formatted output, it would be a lot more robust to verify the data used for that table (and just trust that Rich does the right thing with that data - after all, it comes with its own tests). Besides, isn’t that what you’re really interested in - not whether the script produces a table that looks right, but whether e.g. source-main/.github/workflows/docs.yml is a File of size 1.2KiB (better yet, an actual number of bytes) as expected?

2 Likes
5

That makes so much sense!

You could have found this out by trying the code manually, outside the testing framework.

I actually did, but my terminal was always full screened so I never noticed the ellipsis

Anyway, I was so lost on this. Thank you!