An audio (or multimedia) module

Ideas
1

Maybe I’m being selfish in thinking this since I’m interested in audio-related things, but I think it’d be a good idea to have a module in python for reading/writing several audio formats. Partly, I don’t want to deal with ffmpeg or libsndfile too, as good as they are. A natural extension of this idea would probably include images and videos.

I’m aware PEP 594 got some audio related modules removed, I’m not arguing about bringing all of those back individually. Maybe AIFF could fall under the new module.

I’m also aware tkinter covers some things in image/video, but am unfamiliar with it so I’ll leave discussion to anyone else on that matter.

Maybe Python’s JIT (which I know has a PEP open waiting to be resolved to resume progress on its development) is a factor on how much can be implemented in such a module without relying on any external libraries. Would be interesting if no C/C++ libraries would be used, but I understand that’s not my decision.

A longer-term want is to have some audio backend abstractions as well, but that would probably be much harder to implement compared to audio/image/video format specs that don’t frequently change or haven’t changed in years. Personally, I’d like if it wouldn’t have to rely on a C/C++ library like portaudio/miniaudio, but again not my decision.

What audio formats do I want covered in such a new module? Distinguishing some terms:

  1. Containers/formats: WAV (RIFF), AIFF/AIFC, FLAC (native), CAF, Ogg (Vorbis, Opus, FLAC), MP3 (MPEG-1 Audio Layer III frames), standard MIDI file (SMF).
  2. Codecs: PCM (uncompressed), FLAC, Vorbis, Opus, ALAC, MP3, Microsoft IMA ADPCM.
  3. Maybe cover some ambisonics metadata too? Though I can’t explain since I don’t know it well.

Maybe cover MIDI 1.0/SMF, though that’s not an audio file format. midi could be its own module?

What do I want to be able to do with the audio formats? As the first paragraph mentioned, read, write and also seek, at least. Distinguish between frames and the time in seconds. (And if audio backend abstraction was implemented, play audio by writing less than 5 lines of code.)

Sorry if this seems all over the place or not detailed enough (audio or otherwise), I am not too familiar with the software side of audio/music despite making this idea post. I instead see value in initiating the discussion on having a dedicated audio/multimedia module for dealing with (at least audio) formats that are used day-to-day, and hope something good comes out of it. Obviously the scope looks big so I assume a PEP would have to be made if such an audio/multimedia module needs to come to existence.

Also despite being the one to start the post, I don’t think I can bring enough to the discussion, so I hope other people can take over the discussion and come up with clearer ideas/explanations for/against this. If it makes sense for fewer codecs and formats to be covered in a stdlib module, so be it. Better one post to see discussion on this than none. (I didn’t find any similar post in idea before making this)

Edit: If this reads too much like an “it’d be nice”, I’d like to mention having these read/write capabilities can help with file format conversion, maybe changing sampling rates too, make slight volume adjustments to a file’s contents, maybe other things that one can do in a DAW (but probably not everything one can do in a DAW? That’d be too much.)

2

See the devguide section Adding to the stdlib, in particular the points about adding burden to the core team, and modules needing to be useful to a large portion of the community.

There’s no reason a module like this needs to be part of the standard library. Third party packages can support all of the audio or multimedia needs.

3 Likes
3

There’s nothing wrong with scratching your own itch - that’s how the open source world works!

Have you looked on PyPI for the things you’re looking for? MIDI, for example, is simple enough that there are quite a few packages that work with it, and I’d guess most of them are pure Python.

For actually working with audio samples, though, I think you might run into one fundamental that makes them difficult to work with in the standard library: you would almost certainly want a high performance array of numbers with vectorization tools, so that means that your high level format for audio samples is very likely to be built on top of numpy. That’s perfectly fine for a module on PyPI, but it won’t fly in the stdlib.

1 Like
4

Given that there are capable third-party libraries that you “don’t want to deal with”, what issues are you looking to be addressed by including equivalent functionality in the standard library? Is the hesitation to use them that the packaging makes them difficult to use? Are they more complex than you think is necessary?

Shedding more light on why you don’t want to use the existing third-party libraries and think new ones should be added to the core would help make your case for adding them or (more likely) elicit responses that help you overcome whatever that reluctance is.

1 Like
5

Note that PEP 594 (PEP 594 – Removing dead batteries from the standard library | peps.python.org) removed several multimedia-related modules because they had become obsolete and unmaintained. That doesn’t bode well for adding new similar functionality. PyPI is a better place for this.

3 Likes
6

Ah, looking at the prerequisites I think my perspective moves from a “This kind of module should exist” to “Such a module would have to exist first as a package in PyPI, and chances are I’ll have to write it up myself.” and “Perhaps I was wrong on there being any preexisting demand for such a module to be in the stdlib.”

Not in depth, no. I’m aware of soundfile and python-ffmpeg for the audio format reading and writing, but not beyond.

Good to know actually.

That I’m aware of. It’s looking more and more likely that I’ll have to figure and write the package I want, before getting it to PyPI and eventually, ponder what subset to bring up as an idea for the stdlib.

Here comes what might sound ridiculous: I want to write an audio-related package, but I want to implement such a thing using mostly the stdlib, though having the option to use numpy, and perhaps a handful packages (though trying to keep the count low). Doing my research for that, I saw that only the wav format was covered in the stdlib. (How well covered it is, I don’t know.) From that, I had the idea of starting a discussion to have a module that covers multiple formats.

Additionally, I didn’t want to rely on C/C++ libraries for my project (well, to the extent of things that don’t come with an installation of python), when choosing what packages to use.

Yeah, I am aware. But I was under the impression that the… common-ness of dealing with image, video and audio files might make my idea more legitimate in a long-term view. And was also under the impression that the precedent was a matter of obsoleteness and maintainability.

I suppose some of the questions that remain for discussion is “Is there such a package already that doesn’t rely on additional python packages and is considered best-of-breed?” “Would any subset of what such a package covers be suitable to be covered in the stdlib under a new module?” (Already got one hard no to making a new module, I see.) And “Are the people who currently maintain any such module willing to maintain it for 2 years(?) ?”