Do we still need `Tools/{io,cc,string}bench`?

Core Development
1

There are Tools that look quite unused:

  1. cpython/Tools/iobench at main · python/cpython · GitHub
  2. cpython/Tools/stringbench at main · python/cpython · GitHub original issue: Issue 13165: Integrate stringbench in the Tools directory - Python tracker
  3. cpython/ccbench.py at main · python/cpython · GitHub

They still claim to support python2 and haven’t been touched for over a decade (except some minor global refactorings like https://github.com/python/cpython/commit/8f943ca25732d548cf9f0b0393ba8d582fb93e29)

So, the question is: do we still need them? Aren’t these cases covered by GitHub - python/pyperformance: Python Performance Benchmark Suite ? Are they required for faster-cpython project?

I see several ways forward:

  1. Do nothing, they are good as-is
  2. Modernize them to be fully useful inside Tools/: add new feaftures to the test (like new str methods), improve their docs, maybe mention them in devguide
  3. Move them to pyperformance suite if needed and delete them from Tools/
  4. Just delete them - they are outdated (I doubt that)

CC @pitrou @vstinner @storchaka who worked on them initially.

3 Likes
2

Existing discussion in cpython: Move Benchmarks From Tools/ to the pyperformance Repo · Issue #97680 · python/cpython · GitHub

3

I’m in favor of just deleting them. They’re in version control, if another repo (pyperformance is a good example) wants to adopt any of them for any particular purpose they’ll always exist there.

2 Likes
4

Please see PR #101853 to remove them.

5

Some context:

  • iobench was developed (by me) when we rewrote the Python 3 IO stack in C
  • ccbench was developed (by me) when I rewrote the GIL implementation

None of them are of course necessary to keep, though the maintenance cost must be close to zero.