An audio (or multimedia) module

Ideas
1

Maybe I’m being selfish in thinking this since I’m interested in audio-related things, but I think it’d be a good idea to have a module in python for reading/writing several audio formats. Partly, I don’t want to deal with ffmpeg or libsndfile too, as good as they are. A natural extension of this idea would probably include images and videos.

I’m aware PEP 594 got some audio related modules removed, I’m not arguing about bringing all of those back individually. Maybe AIFF could fall under the new module.

I’m also aware tkinter covers some things in image/video, but am unfamiliar with it so I’ll leave discussion to anyone else on that matter.

Maybe Python’s JIT (which I know has a PEP open waiting to be resolved to resume progress on its development) is a factor on how much can be implemented in such a module without relying on any external libraries. Would be interesting if no C/C++ libraries would be used, but I understand that’s not my decision.

A longer-term want is to have some audio backend abstractions as well, but that would probably be much harder to implement compared to audio/image/video format specs that don’t frequently change or haven’t changed in years. Personally, I’d like if it wouldn’t have to rely on a C/C++ library like portaudio/miniaudio, but again not my decision.

What audio formats do I want covered in such a new module? Distinguishing some terms:

  1. Containers/formats: WAV (RIFF), AIFF/AIFC, FLAC (native), CAF, Ogg (Vorbis, Opus, FLAC), MP3 (MPEG-1 Audio Layer III frames), standard MIDI file (SMF).
  2. Codecs: PCM (uncompressed), FLAC, Vorbis, Opus, ALAC, MP3, Microsoft IMA ADPCM.
  3. Maybe cover some ambisonics metadata too? Though I can’t explain since I don’t know it well.

Maybe cover MIDI 1.0/SMF, though that’s not an audio file format. midi could be its own module?

What do I want to be able to do with the audio formats? As the first paragraph mentioned, read, write and also seek, at least. Distinguish between frames and the time in seconds. (And if audio backend abstraction was implemented, play audio by writing less than 5 lines of code.)

Sorry if this seems all over the place or not detailed enough (audio or otherwise), I am not too familiar with the software side of audio/music despite making this idea post. I instead see value in initiating the discussion on having a dedicated audio/multimedia module for dealing with (at least audio) formats that are used day-to-day, and hope something good comes out of it. Obviously the scope looks big so I assume a PEP would have to be made if such an audio/multimedia module needs to come to existence.

Also despite being the one to start the post, I don’t think I can bring enough to the discussion, so I hope other people can take over the discussion and come up with clearer ideas/explanations for/against this. If it makes sense for fewer codecs and formats to be covered in a stdlib module, so be it. Better one post to see discussion on this than none. (I didn’t find any similar post in idea before making this)

Edit: If this reads too much like an “it’d be nice”, I’d like to mention having these read/write capabilities can help with file format conversion, maybe changing sampling rates too, make slight volume adjustments to a file’s contents, maybe other things that one can do in a DAW (but probably not everything one can do in a DAW? That’d be too much.)

2

See the devguide section Adding to the stdlib, in particular the points about adding burden to the core team, and modules needing to be useful to a large portion of the community.

There’s no reason a module like this needs to be part of the standard library. Third party packages can support all of the audio or multimedia needs.

8 Likes
3

There’s nothing wrong with scratching your own itch - that’s how the open source world works!

Have you looked on PyPI for the things you’re looking for? MIDI, for example, is simple enough that there are quite a few packages that work with it, and I’d guess most of them are pure Python.

For actually working with audio samples, though, I think you might run into one fundamental that makes them difficult to work with in the standard library: you would almost certainly want a high performance array of numbers with vectorization tools, so that means that your high level format for audio samples is very likely to be built on top of numpy. That’s perfectly fine for a module on PyPI, but it won’t fly in the stdlib.

1 Like
4

Given that there are capable third-party libraries that you “don’t want to deal with”, what issues are you looking to be addressed by including equivalent functionality in the standard library? Is the hesitation to use them that the packaging makes them difficult to use? Are they more complex than you think is necessary?

Shedding more light on why you don’t want to use the existing third-party libraries and think new ones should be added to the core would help make your case for adding them or (more likely) elicit responses that help you overcome whatever that reluctance is.

2 Likes
5

Note that PEP 594 (PEP 594 – Removing dead batteries from the standard library | peps.python.org) removed several multimedia-related modules because they had become obsolete and unmaintained. That doesn’t bode well for adding new similar functionality. PyPI is a better place for this.

4 Likes
6

Ah, looking at the prerequisites I think my perspective moves from a “This kind of module should exist” to “Such a module would have to exist first as a package in PyPI, and chances are I’ll have to write it up myself.” and “Perhaps I was wrong on there being any preexisting demand for such a module to be in the stdlib.”

Not in depth, no. I’m aware of soundfile and python-ffmpeg for the audio format reading and writing, but not beyond.

Good to know actually.

That I’m aware of. It’s looking more and more likely that I’ll have to figure and write the package I want, before getting it to PyPI and eventually, ponder what subset to bring up as an idea for the stdlib.

Here comes what might sound ridiculous: I want to write an audio-related package, but I want to implement such a thing using mostly the stdlib, though having the option to use numpy, and perhaps a handful packages (though trying to keep the count low). Doing my research for that, I saw that only the wav format was covered in the stdlib. (How well covered it is, I don’t know.) From that, I had the idea of starting a discussion to have a module that covers multiple formats.

Additionally, I didn’t want to rely on C/C++ libraries for my project (well, to the extent of things that don’t come with an installation of python), when choosing what packages to use.

Yeah, I am aware. But I was under the impression that the… common-ness of dealing with image, video and audio files might make my idea more legitimate in a long-term view. And was also under the impression that the precedent was a matter of obsoleteness and maintainability.

I suppose some of the questions that remain for discussion is “Is there such a package already that doesn’t rely on additional python packages and is considered best-of-breed?” “Would any subset of what such a package covers be suitable to be covered in the stdlib under a new module?” (Already got one hard no to making a new module, I see.) And “Are the people who currently maintain any such module willing to maintain it for 2 years(?) ?”

7

I really want to understand: why are you willing to use numpy (a third-party package), and maybe a few others, but don’t want the audio foundations handled by a third-party package?

8

Agreed. The packaging community in particular is very aware that there are people who for various reasons cannot use 3rd party packages. But this is the first time I’ve heard of someone who is willing to use some 3rd party packages but not others.

Is it because numpy has some sort of “official status” because of its central role in the scientific community? If so, then what would give the same legitimacy to music processing libraries? In the absence of some sort of “music processing with Python” community I’m not sure who would have the expertise to make such judgements.

If the problem is simply that you don’t know how to find good music processing libraries, then that’s a reasonable complaint. PyPI doesn’t really offer that sort of discovery service, and no-one else has really tried to provide it (there are various “Awesome Python Libraries” style sites, I believe, but I haven’t tried them so I can’t comment on their value). That would be a good (although probably difficult!) community project, if anyone was interested in working on it.

4 Likes
9

I personally don’t want to rely on a C/C++ library, directly or indirectly, to handle audio formats in my python project, more or less. Of course if somehow python were to get a new module for audio or such, and it depended on a C/C++ library, I’d have to consider whether/how my position’s wording changes, but that’s a very big if currently.

(And I’d also have to take into account library licenses. I want to use MPL-2.0. I know using packages relying on GPL/LGPL code makes compliance intimidating or unfeasible.)

NumPy in some sense is an exception for my no-3rd party packages because I can imagine requests for using it as a backend for …computations(?) over just stdlib in my project.

I wouldn’t put my decision as it being “official” but rather it being “proven/known to be (probably) best for computation in some regard”. Though the question of having a music processing library having a similar legitimacy is a very interesting question to me. (But I’m not convinced/aware there’s merely one - three or four packages that cover everything I need and aren’t tied to C/C++ libraries or don’t have significant overlap, so not my place to speak on what may be “official”?)

10

But Fortran is okay? I’m still not sure what the logic here is.

1 Like
11

But why? Python itself is written in C, Numpy is written in C, plus Fortran, and possibly C++. And if someone added an audio format processing library to the stdlib, it could easily be written in C (and likely would be, if performance was important which I imagine it would be).

I’m still unclear what the boundaries you’re setting are.

1 Like
12

The boundary is C/C++ audio libraries and MPL-2.0 incompatible libraries/packages for myself. Perhaps I should’ve made that clearer earlier, sorry.

13

OK. I think you’re going to have to write the code yourself, then. I can’t imagine any library maintainer being willing to stick to those guarantees (even if they do right now, adding a bit of C/C++ to improve performance at a later date seems like a pretty reasonable implementation choice to make).

2 Likes
14

The license on your libraries shouldn’t be a problem normally; your code isn’t required to be GPL’d just because it calls on a GPL library.

15

This still isnt’ clear. Why are C audio libraries no-go, but C libraries for other parts of your code are OK?

1 Like
16

While I think some of the prior removals may not have been needed (code not recieving changes doesn’t mean dead battery in all cases; in many cases, it just means there’s no maintenance required, code is allowed to reach a finished state…), I don’t think undoing those removals is going to help at this point.

Your specific requirements are pretty strange. Does the requirement of not having any C/C++ include projects that originated within python and are the community supporting the removed packages (eg. audioop-lts)?

You can depend on GPL libraries while still licensing a project under MPL-2.0, so either the whole picture hasn’t been explained, or this is just a misunderstanding (in an admittedly a hard subject with a ton of variance and legal implications attached)

17

There are some chances that C libraries exist that fit my needs, that don’t overlap with audio-specific functionalities/algorithms. I left the window open for a decision for my future self in that regard.

I wasn’t implying undoing everything, it just happens to be that aiff/aifc might fall under the (hypothetical) new module’s scope. If your concern is related to the format’s relevance, that is noted.

I’m inclined to act so, at least for the sake of minimizing external dependencies.

Perhaps I’m getting very off-topic now, but I’m intimidated by legal compliance in the case of not labelling my project as GPL after depending on GPL library/libraries.

I’m aware Python allows C/C++ extending and embedding, and don’t disagree on the last bit. That being said, you’re probably right that I’d have to write the code myself and I should start planning in detail the first 2-3 projects I first want to attempt to make.

19

I would use this every day.

But I think it’s a bad idea to put it into the standard library.

Use soundfile and sounddevice.

20

IANAL, but my understanding of GPL is that practically speaking if your code touches GPL code (like basically “runs in the same process”) it becomes a derivative work and is required to abide by the GPL’s copyleft requirements. LGPL provides allowance for dynamic linking of third-party libraries without infecting your code with copyleft requirements.