This is a straightforward structural proposal: PEP 799 introduces a new profilers standard library module to house Python’s built-in profilers under a single, coherent namespace.
It:
Adds profilers.tracing (alias for cProfile)
Adds profilers.sampling ( with the code for tachyon, the new sampling profiler in 3.15 using the introspection technology we added in 3.14)
Deprecates the legacy profile module
This should make profiler discovery easier, reduce confusion, and set us up for clean future extensions (memory profilers, I/O profilers, etc.).
why is it important for this functionality to be stdlib compared to a PyPI package? (bearing in mind that subinterpreters were suitable for PyPI, despite being entirely a core runtime feature)
we’ll have compression and concurrent namespaces - profilers language-wise stands out. Why not profiling for consistency? (Arguably reusing profile would be more consistent, but too ambiguous IMHO)
Sampling profilers like this require a extremelly tight coupling with CPython internals. They depend on accessing low-level execution state with minimal overhead, supporting multi-threaded programs, free-threading, async code, and even remote attachment. All of this requires syncing the profilers AND the features, even between patch versions. Because of this deep integration, changes in CPython internals can easily break or degrade sampling profilers (this happens constantly and is very painful for everyone). Keeping it in the stdlib ensures synchronization with those changes.
Having the profiler in the stdlib ensures users have a high performance profiler with these features working on the release of CPython and they don’t need to wait months or years until the other tools reverse engineer all the ways CPython broke them or make their life harder.
We can use the profiler to profile Python code during development of CPython. Normally not having into the sodlib means no profiler works until the release so we cannot benefit from it during development cycles.
The stdlib now lacks a performant profiler that works for the different things that we offer (free threading, asyncio, …etc). Including a well-supported, zero-overhead profiler in the stdlib provides a known baseline and reference implementation that other tools can use as a baseline or emulate. It also avoids fragmentation, which is especially important in performance tooling. The PEP goes into some more detail.
It also provides a stable, officially supported reference implementation that external tools can study or build on without needing to resort to undocumented or fragile techniques. This will help everyone know a baseline on how to adapt their things to new versions and don’t force them to have to find out contorted ways to keep the use case working and ensures that we don’t break these tools on every release.
Including this in the stdlib helps steer users away from outdated tools like profile, and toward modern, efficient practices.
I don’t mind that If everyone else agrees. How strongly do you feel about this? One downside of this is that profiling exists on PyPI but profilers doesn’t.
(Arguably reusing profile would be more consistent, but too ambiguous IMHO)
That would be super confusing for everyone and make the deprecation much harder IMHO
I agree that profiling is a better module name than profilers. The profiling package on PyPI hadn’t been updated since 2017, and its GitHub repo is archived, so I don’t see that as a strong argument against that name.
I have a nitpick: do we really need the sampling profiler available under two names, profilers.sampling and profilers.tachyon? Aliases can make sense when we need compatibility, but if we’re introducing something new, I’d prefer having it under only one name so users don’t get confused.
Need? no. The idea is that we would mostly advertise the profilers.sampling one but I wanted to have an alias for the name the same way you would have cProfile (which we are not removing) and profilers.tracing. I can be persuaded to remove the alias and just add it to the docs somehow in a way that the docs working group doesn’t find very confusing if you feel very strongly about it.
Is the intent to move cProfile.py to profiling/tracing.py? The PEP notes that there will be an ‘alias’ from one to the other and also that profilers.tracing will become the promoted name from 3.15. Moving things so that cProfile.py becomes the alias/reexport as the deprecated location would seem to make sense to me, and means that the sources for all profiling modules would be in the same place. This would also reduce the need for any new aliases (eg tachyon) — though happy to have a mention of this codename in the docs
This doesn’t matter for the PEP as is not user visible but the plan is just to alias it. The original module remains the same. This is mostly to reduce moving things but the users wouldn’t care where the actual code lives. The new and main advertised module will be profilers.tracing
Hi there. I’m the author of Austin. First of all thanks @pablogsal for this proposal! For what my opinion might be worth, I think having at the very least the core functionalities that power a statistical profiler in the stdlib is beneficial for observability in general. This would make the life of e.g. continuous profiler maintainers much easier. As for end users, having a tool already available with a standard CPython installation might also be a benefit as they can profile their code straight-away without any extra dependencies, with perhaps the extra guarantees (e.g. on accuracy) of a tool that has been developed by the code developers themselves. And as the maintainer of a third-party tool, this might also relieve me from the “burden” of having to maintain my own tooling to fill in the current gap.
With most of the technology that has been developed for tools like Austin making its way into the stdlib there might be a unified effort to make just a single tool better, rather than have multiple parallel efforts split across different projects. I believe other communities have already taken this approach (I think Golang has many baked-in profiling solutions these days).
Good point. I am torn about this because on one side one could think that the best approach for backwards compatibility is to leave the current code where it is and create aliases just to protect for the very unlikely case of some user relying on __qualname__ to not change.
On the other hand we could follow what @AA-Turner proposes and just move everything under the new namespace and reexport the old name as an alias to keep things consistent and ensure we advertise the new namespace.
If I were to become a (new) profilers maintainer (won’t happen), I am sure I would prefer everything in one directory, and not have to know the aliases. As a triager that occasionally peeks a C code, I would now prefer to find it under the preferred name and ‘forget’ ‘cProfile’. This will become more true in future years.
It looks great to me for building a dedicated profiling package.
On the other hand we could follow what @AA-Turner proposes and just move everything under the new namespace and reexport the old name as an alias to keep things consistent and ensure we advertise the new namespace.
I prefer this action. I think we need to keep the old package for one or two major version at least to allow people to migrate their code.
Note that the PEP does not propose deprecating or removing the cProfile module/name; both cProfile and profiling.tracing will point to the standard library’s tracing profiler.
Yes, I believe we have the same thought. We move the old cProfile module to profiling.tracing and alias profiling.tracing as cProfile
The reason I mention PEP 387 in the last comment base on the reason following below:
I believe we will remove cProfile module in the future to keep the stb lib clean and there would be just one profiling lib here.
I have noticed that @pablogsal has documented the profile lib deprecation timeline as following below:
In Python 3.15: importing profile emits a DeprecationWarning.
In Python 3.16: all uses of profile emit a DeprecationWarning.
In Python 3.17: the module will be removed from the standard library.
But I’m not sure whether we need to discuss what we need to deprecate the cProfile in the certainly future and whether we need to define a deprecation timeline just like we did for profile lib.
Very unlikely. This will break users for not a lot of benefit. For example removing the aliases from collections.abc was quite painful and I don’t want to inflict a similar situation to users if we can avoid it. We can surely deal with some aliases in the stdlib. We can remove the references from the docs and eventually people will stop using it.
The difference here is that cProfile is still usefull as a tracing profiler (maybe less usefull than it used to be due to the recent optimizations and how they behave in the presence of profiling functions) so it doesn;t make sense to deprecate. profile has no advantage and is just adding confusion to users.
/ 
Hi there. I’m the author of