# Why pytest is not a battery? **URL:** \ **Category:** Python Help\ **Tags:** stdlib\ **Created:** [February 28, 2023, 6:20pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331 "2023-02-28T18:20:35Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Marco\_Sulla](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/marco_sulla/32/1451_2.png) [@Marco\_Sulla](https://discuss.python.org/u/Marco_Sulla)\ **Post date:** [February 28, 2023, 6:20pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/1 "2023-02-28T18:20:35Z")
I find that pytest is really handy and powerful test tool. But it’s not in the py stdlib. Is it because it changes the behavior of `assert`? ---
**Author:** ![ajoino](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ajoino/32/6907_2.png) [@ajoino](https://discuss.python.org/u/ajoino)\ **Post date:** [February 28, 2023, 6:42pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/2 "2023-02-28T18:42:06Z")
There are many possible reasons, for pytest in particular I’d think a combination of the following: - the unittest module exists, and the stdlib doesn’t need two “competing” libraries. - If a library gets added to the stdlib it essentially gets “frozen” in a state where each addition and change and their consequences must be carefully considered. I haven’t followed the development of pytest that much but I reckon many of the features wouldn’t be there if it beeded to be more stable. - Someone correct me if I’m wrong, but I assume that none of the maintainers of pytest have offered to put it in the stdlib. Compare this to the addition of the tomllib module, where the maintainer was asked and was willing to put the effort of porting the project to the stdlib. This includes (co-)authoring a pep and participate in discussions, which can be mentally draining. ---
**Author:** ![Marco\_Sulla](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/marco_sulla/32/1451_2.png) [@Marco\_Sulla](https://discuss.python.org/u/Marco_Sulla)\ **Post date:** [February 28, 2023, 7:32pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/3 "2023-02-28T19:32:10Z")
> [@ajoino](#): > > the unittest module exists, and the stdlib doesn’t need two “competing” libraries. Well, there’s a precedent: pathlib and os.path > [@ajoino](#): > > I haven’t followed the development of pytest that m7ch but I reckon many of the features wouldn’t be there if it beeded to be more stable. Indeed I surfed on the pytest release notes page and it’s full of “breaking changes” announcements. > [@ajoino](#): > > Someone correct me if I’m wrong, but I assume that none of the maintainers of pytest have offered to put it in the stdlib. Compare this to the addition of the tomllib module, where the maintainer was asked I really don’t know if Python asked pytest team to add the module in the stdlib. ---
**Author:** ![smontanaro](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/smontanaro/32/1389_2.png) [@smontanaro](https://discuss.python.org/u/smontanaro)\ **Post date:** [February 28, 2023, 9:05pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/4 "2023-02-28T21:05:10Z")
In addition to the points Jacob already made: - In today’s virtual environment world, you probably only need pytest during development, only in your build time virtual environments. It will just get downloaded and used when and where it’s needed. - I know it’s been more-or-less separated from the original py package, but are there still dependencies on that? If so, it might well bring along a lot of extra baggage that isn’t necessarily suitable for the standard library. - Some core dev(s) has to take ownership of it (or a pytest maintainer has to become a core dev). Many of the points made in this thread would apply when considering other popular packages for inclusion in the standard library. ---
**Author:** ![cameron](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cameron/32/2658_2.png) [@cameron](https://discuss.python.org/u/cameron)\ **Post date:** [February 28, 2023, 10:12pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/5 "2023-02-28T22:12:50Z")
> > [@ajoino](#): > > > > the unittest module exists, and the stdlib doesn’t need two “competing” libraries. > > Well, there’s a precedent: pathlib and os.path Pathlib is very recent, and represents a significant change to pocessing filesystem paths, at least in terms of notation. More to the point, there are several “more than unittest” test finders/runners. pytest does a lot, and so to the others. Differently. Putting eg pytest in the stdlib preferences that test kit over the others in a sense. `unittest` is there as a basis, and most others build off it or replicate its fucntionality. `unittest` does grow more assertion types of time. > > [@ajoino](#): > > > > I haven’t followed the development of pytest that m7ch but I reckon many of the features wouldn’t be there if it beeded to be more stable. > > Indeed I surfed on the pytest release notes page and it’s full of “breaking changes” announcements. Yeah. We don’t do that in the stdlib very much. Nearly never. Modules in the stdlib need to be _very_ forward compatible. People _need_ the stdlib to just continue working with newer python releases. Also, stdlib module updates _only_ happen with Python releases. And the flip side is: they happen coercisvely with a Python release. Using a newer Python? You’re using the newer stdlib modules. With a third party library you can pin to a particular release, which can be important if you haven’t updated for some breaking change. It’s also important for reproducability. Also, everything in the stdlib is a maintenance burden for a limited set of people. The third party libraries have their own authors/teams. Broadly, it’s usually a good thing for stuff beyond the basics to be third party libraries. Cheers, Cameron Simpson [cs@cskk.id.au](mailto:cs@cskk.id.au) ---
**Author:** ![Marco\_Sulla](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/marco_sulla/32/1451_2.png) [@Marco\_Sulla](https://discuss.python.org/u/Marco_Sulla)\ **Post date:** [March 2, 2023, 8:24pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/6 "2023-03-02T20:24:26Z")
> [@smontanaro](#): > > I know it’s been more-or-less separated from the original py package, but are there still dependencies on that? Yes, a lot… I understood `pytest` can’t join the stdlib for too much reasons you all pointed out. I hope someday `pytest` will overcome these problems, or `unittest` will incorporate some of the `pytest` features. The replies after this point is only for the sake of arguing, not to encourage to put pytest in the stdlib. * * * * * * > [@smontanaro](#): > > In today’s virtual environment world, you probably only need pytest during development, only in your build time virtual environments. It will just get downloaded and used when and where it’s needed. Not sure to understand. Also `unittest` is used for development. > [@cameron](#): > > Pathlib is very recent I’m not sure you can define Python 3.4 “recent”. Furthermore, a new module hardly is mature enough to be a stdlib candidate. > [@cameron](#): > > Putting eg pytest in the stdlib preferences that test kit over the > others in a sense. Well, of course. If you choose to integrate a module in the stdlib, you clearly prefer it over the others. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [March 2, 2023, 8:46pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/7 "2023-03-02T20:46:28Z")
> [@](#): > > I hope someday `pytest` will overcome these problems, or `unittest` will incorporate some of the `pytest` features. They are not “problems”, they are _features_. Third-party libraries can innovate as much as they want, they can change their own APIs whenever they want, they can release whenever they feel like it. These aren’t bad things to be overcome, they are positively good things for third-party libraries (especially when they are young and experimental). unittest’s fundamental design is very different from pytest’s design. They work in very different ways. What sort of features do you think it can copy from pytest? ---
**Author:** ![Marco\_Sulla](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/marco_sulla/32/1451_2.png) [@Marco\_Sulla](https://discuss.python.org/u/Marco_Sulla)\ **Post date:** [March 2, 2023, 10:28pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/8 "2023-03-02T22:28:51Z")
> [@steven.daprano](#): > > They are not “problems”, they are _features_. Good point. > [@steven.daprano](#): > > What sort of features do you think it can copy from pytest? Not exhaustive list: - simple tests can be functions - simple fixture decorator - parametrize - simplified discovery with `test_` prefix - more detailed and knowable output And I like also `assert x == 10` vs `self.assertEqual(x, 10)` etc, but I don’t think it will be ever in the language core. ---
**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\ **Post date:** [March 3, 2023, 9:14am UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/9 "2023-03-03T09:14:58Z")
Disclaimer: my pytest experience is microscopic. Suggested features: > [@](#): > > - simple tests can be functions > > - simple fixture decorator > > - parametrize > > - simplified discovery with `test_` prefix > > - more detailed and knowable output In unittest simple tests are methods. In the simplest case of all, that just adds one line and one indent to your test function: ```auto class MyTest(unittest.TestCase): def test(self): self.assertTrue("this test passes") ``` unittest already has test discovery. I don’t know what “more detailed and knowable output” means to you. As far as parametized tests go, I have found the easiest way is to put the test in a separate method and the parameter handling in the test method: ```auto class MyTest(unittest.TestCase): def check_something(self, arg): self.testEqual(arg, "something") def test(self): for param in things: self.check_something(myfunc(param)) ``` ---
**Author:** ![ajoino](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ajoino/32/6907_2.png) [@ajoino](https://discuss.python.org/u/ajoino)\ **Post date:** [March 3, 2023, 9:24am UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/10 "2023-03-03T09:24:36Z")
@steven.daprano Will your parametrization show, in the output, as one test per parameter, or one test for all parameters? Pytest shows one test per parameter and that’s what I assume the question about parametrization was about. ---
**Author:** ![Marco\_Sulla](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/marco_sulla/32/1451_2.png) [@Marco\_Sulla](https://discuss.python.org/u/Marco_Sulla)\ **Post date:** [March 3, 2023, 7:12pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/11 "2023-03-03T19:12:33Z")
> [@steven.daprano](#): > > Disclaimer: my pytest experience is microscopic. Don’t worry, not an expert here X-D > [@steven.daprano](#): > > In unittest simple tests are methods. In the simplest case of all, that just adds one line and one indent to your test function: > > ```auto > class MyTest(unittest.TestCase): > def test(self): > self.assertTrue("this test passes") > > ``` in pytest, this is: ```auto def test_1(): assert "this test passes" ``` > [@steven.daprano](#): > > unittest already has test discovery. Yes, but with pytest is quite more simple. You have just to start your tests files and methods with `test_`. > [@steven.daprano](#): > > I don’t know what “more detailed and knowable output” means to you. Mh, I think you should simply run a pytest example and see the difference. > [@ajoino](#): > > Pytest shows one test per parameter I add that it’s quite more simple to write, since it uses a decorator. ---
**Author:** ![PythonCHB](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pythonchb/32/2551_2.png) [@PythonCHB](https://discuss.python.org/u/PythonCHB)\ **Post date:** [March 11, 2023, 2:10am UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/12 "2023-03-11T02:10:53Z")
### My personal take: unitest is an old and klunky and not very “pythonic” test framework. It was ported from Java, so that’s not a surprise. One of these days, I’ll write a more exhaustive review, but I’ll leave it at that for now. So I don’t recommend it to folks looking to set up tests for their own new projects. And if you look at large third party projects – a huge number use pytest (and nose before that), or maybe some others, I’m not sure. I don’t mean to be critical or second-guess its original inclusion in the stdlib – at the time Unit Testing was pretty new, and it was a very good idea to get a test framework into the stdlib. junit was popular and successful – much better to bring in something that was known to work than to start from scratch. And it’s in the stdlib because it’s used to test the stdlib itself – that’s why it’s not simply a third party library. ### Should pytest be included in the stdlib? I think not – it’s a large and powerful test framework, and I think everyone should use it (at least in preference to unittest). But you really can’t get far with Python without using SOME third party package. Particularly for testing, it’s a very small lift to require pytest to run your tests – so no need to put it in the stdlib. But as unittest is in the stdlib, it is often seen as the “standard” or “official” (and implicitly the best) way to write tests, which is too bad. And unitest is the only option for testing the stdlib itself. ### My (half baked) Proposal: 1. Soft-deprecate `unittest`: While it should never be removed, we should stop recommending it as the “standard” way to write tests – even implicitly. 2. Add a “pytest light” to the stdlib: pytest has a LOT of features and complexity – and it continues to evolve - moving too fast for the stdlib. But it would help a lot to have a simpler, more “pythonic” test framework in the stdlib – for use by the stdlib itself, and as a starting point for folks to write their own tests. A “pytest light” may or may not use any pytest code, but it would be a pytest compatible subset of python. So any tests written for pytest light would run under pytest – that way folks could easily move to pytest to get more features as their testing needs got more complex. Much like you can now use pytest to run your unittest tests out of the box. So what would a pytest light include? Here’s my off the top of my head personal list, based on what I commonly use: #### 1 implicit test failure messages This is THE BIG ONE – pytest uses fancy introspection tricks to tell you why a test failed – it is incredibly useful, and completely removes the need for “special asserts” – just write: ```python assert result == expected ``` and you are done. You can even embed a function call and get a good error message: ```python assert math.isclose(result, expected) ``` So no need for special asserts #### 2 No need for testcase classes Any function can be a test: ```python def test_this(): [some setup] assert something ``` and you have a test. ( you can still make test classes if you like) #### 3 Parametrized tests Very easy to write tests with a bunch of variied input – and they show up as separate tests. #### 4 function and generator based fixtures. This goes with the no test classes: ```python @pytest.fixture def my_fixture(): [do some setup] yield the_setup_thing [do some teardown] def test_something_with_a_fixture(my_fixture): ... result = something(my_fixture) ... assert result == expected ``` Simple and very flexible. #### 5 contex tmanager for testing exceptions ```python with pytest.raises(ZeroDivisionError): 3 / 0 ``` That’s my 5 feature list – I’m sure others have other opinions about what’s most important, and once we included everyone’s ideas, we may end up with all of pytest – but I think a simple version would be very helpful – even only the first two on that list would be great. ---
**Author:** ![smontanaro](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/smontanaro/32/1389_2.png) [@smontanaro](https://discuss.python.org/u/smontanaro)\ **Post date:** [March 11, 2023, 10:55am UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/13 "2023-03-11T10:55:50Z")
> [@PythonCHB](#): > > #### 3 Parametrized tests > > Very easy to write tests with a bunch of varied input – and they show up as separate tests. This was new to me, so I went looking for how to use the feature. And, I thought you’d misspelled “parameterized,” but when I checked the pytest docs I saw it’s misspelled there, quite consistently, both in code and prose. Is there something I’m missing? If pytest is so fluid, can’t they fix their spelling errors? In this case they could provide a correctly spelled attribute then downplay the incorrect spelling, but leave it in the code to avoid breakage. ---
**Author:** ![barneygale](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barneygale/32/1882_2.png) [@barneygale](https://discuss.python.org/u/barneygale)\ **Post date:** [March 11, 2023, 6:32pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/14 "2023-03-11T18:32:14Z")
> [@smontanaro](#): > > And, I thought you’d misspelled “parameterized,” but when I checked the pytest docs I saw it’s misspelled there, quite consistently, both in code and prose. Is there something I’m missing? It’s not misspelt, it’s an [alternative spelling](https://en.wiktionary.org/wiki/parametrize)! ---
**Author:** ![PythonCHB](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pythonchb/32/2551_2.png) [@PythonCHB](https://discuss.python.org/u/PythonCHB)\ **Post date:** [March 11, 2023, 6:39pm UTC](https://discuss.python.org/t/why-pytest-is-not-a-battery/24331/15 "2023-03-11T18:39:52Z")
Did you mean to add a winky-face? Maybe putting a pytest-light into the std lib would be a way to enshrine the “correct” spelling 😉 – best argument for it yet!