Sure, if we call that ML doing “arithmetic” — it’s apparently helpful, yes.
Agreed. BTW, this will go far beyond of the struct module. The memoryview and the array share same inextensible scheme of working with data types.
I believe you are misinformed here. There is no support for bfloat16 in recent drafts of upcoming C standard. Nor in published standards. C and C++ — different languages.
You did fine! You’re right that the fundamental mission of struct (and also, but to a lesser extent, of array) is to interoperate smoothly with C (and I’ll skip repeating C++) storage formats
Not from me. Just be aware of that Python has never been “a leader” in anything involving numerics, but “a follower”, and even then usually lags C by some years. That Python caught on in machine learning wasn’t aimed at, it “just happened”, thanks mostly to heroic efforts by many people outside CPython core development.
What you’re asking for here is, objectively, minor work. Just dealing with CPython’s development work flow is much harder than the work itself .
the only benefit I’m aware of for this would be gpu heavy code, which will already require specialized libraries. The memory benefit this exists for would be thrown out the window by turning it into a pyfloat.
every other numerical case won’t accept the tradeoff that bfloat16 exists to serve.
I think it’s fine to leave this entirely to the various GPU-accelerated ML libraries out there.
Hmm. It is mainly for building up structures, as the name suggests. I’m not sure what you would need of that for calls to C, but it can probably be done better with Cython.
The struct module is good for working with files on disk and structures in memory, where something is packed into a specific arrangement. Less good for arithmetic.
Specifically C "striuct"s. That’s why it has such elaborate support for trying to match the platform C’s endianness and padding decisions for C structs, when that’s what you want.- and why “native mode” (act like the platform C compiler) is the default.
In “native mode” it’s perfect for apps that need to work seamlessly on C’s data representations. In other modes it’s great for platform-independent transmission of simple types (like “network format”).
I don’t believe that’s part of what the OP is aiming at. Python doesn’t support 4-byte float arithmetic either, just the 4-byte float storage format, converting to/from the 8-byte float format as needed but only via struct/array/memoryview operations. All arithmetic on floats of all kinds is done with Python’s floats (C’s double type).
Yes. So is, e.g., doing int arithmetic on bytes. GPUs already support arithmetic on the type, and the OP is right that Intel is adding direct HW support in the x86-64 architectures, eventually planned to end up in consumer chips too. Note that Google invented this type, so it will get as much attention as countless billions of dollars can buy
Sure! Another reason for why a PEP would be essential.
Insufficient interest here to justify burning time searching through docs that aren’t designed to be searched. But I find plenty of strongly suggestive stuff searching for
std::bfloat16_t
instead. Like here:
It doesn’t really matter that C and C++ are different languages. They always move in lockstep on this kind of thing.
In context, I was pushing back against the OP’s hope that they could ignore array at first becausearray doesn’t support e. Which it doesn’t - yet. But that’s temporary, and array (IMO) will support e by the time a PR for any new idea is ready. That’s a de facto raising of the current bar (although really more that array’s failing to support e didn’t meet the de facto bar height - that was a glaring exception to “one set of type codes to rule them all”)..
It’s strange to ignore array on slightly other ground: the new type looks more relevant rather for homogeneous data.
BTW, it one direction — conversion looks trivial, thanks to the existing 'f' type code: just pad with zeros. In another — not. Though, in the ML context I wouldn’t be surprised if people don’t care about incorrect rounding: then just skip two bytes, in turn
I will not ignore array but rather will support it in the first PR. I’ll wait until the e support is added. I appreciate the advice and your taking time to let me know about the work already in progress.
Yes, widening conversion will be trivial, as you noted. For narrowing, rounding to nearest (with ties to even) is valuable (including in the ML context) since bfloat16 doesn’t have much precision. It’s not a good idea to waste information by just discarding the last two bytes of a 32-bit float. Doing so has been shown to affect the overall measure-of-goodness of the ML task. (So, you’re right not to be surprised–they did try it!)
Intel added AVX-NE-CONVERT instructions to perform float32 to bfloat16 round to nearest even to the AVX instruction set (256-bit SIMD) in processors available in 2024. AVX10 is a new versioned naming scheme. AVX10.1 is basically AVX512. The bfloat16 arithmetic instructions are in AVX10.2.
That was my first thought too, but I don’t work in ay relevant field, and am happy to take it on trust that the OP has good reasons for not caring about array for their work.
8->2 byte float is non-trivial in an absolute sense, but not in a relative sense in context. CPython already has code to round and deal with denormals (etc) for two other kinds of narrower floats. bfloat16 doesn’t present any new puzzles compared to that - it’s just the IEEE single format with 16 fewer significand bits.
It seems that NumPy uses “Zf” and “Zd” for complex floating-point single and double precision rather than "F” and "D”. Is that a bug? What should be the complex version of "e”?
From the table, also “2g” for complex “long double” instead of “G”.
That’s their call to make, but I wouldn’t say so. They’re under no obligation to duplicate CPython’s 1-character type codes in all contexts, especially since CPython’s codes have already expanded to exceed what’s clear for 1-character codes to express.
For numpy, looks like “2e” is their obvious choice.
Note, that I was talking about conversion of bfloat16 to/from float32. “To” is trivial, while “From” is not, unless ML people will be happy with bad rounding. But it doesn’t seems to be the case as OP said:
Sure, it’s not hard to adapt the PyFloat_Pack/Unpack2.
The NumPy also uses "F" and "D" character codes for scalars:
Well, there is no native complex type with half-floats in components, at least on level of the C standard and IEC standards for floating-point arithmetic.
I don’t think we should add it just because there is support for half-floats. But if so, probably we should use the 'E' type code — that’s an easy part of decision.
According to the standard, memory layout must be compatible with 2-elements array of appropriate type:
Each complex type has the same representation and alignment requirements as an array type
containing exactly two elements of the corresponding real type; the first element is equal to the real part, and the second element to the imaginary part, of the complex number.
It’s the ctypes module where such a picture might be broken, as above doesn’t say anything about directly passing a complex as argument or return value to a function call. That’s why complex support in the ctypes is not unconditional.
I think we use same convention as NumPy for scalar types. At least I don’t remember differences.
Is that relevant to CPython? Nobody is proposing to add a 32-bit or 16-bit float type to CPython, just type codes to specify storage format in the few contexts that care about that. Whether you read a 64-bit, 32-bit, or 16-bit float via struct (or any other way), you get a Python float back - a 64-bit float. Likewise for storing.- in which case the packer is given a Python (64-bit) float to convert and store.
I just can’t picture a way to ask CPython to convert float16 to or from float32 directly. If that’s so, no code is needed for doing so Only code to convert between 64-bit floats and narrower formats.
Rather to the OP goal of support for bfloat16 conversion to/from CPython’s floats.
The struct module already has 'f' type code. So, conversion from/to float32 is supported. With this, conversion from the bfloat16 is trivial: pad 2-byte value with zeros and treat then such 4-bytes as a float32.
Conversion to is similar: convert to float32, then drop 2 bytes. Of course, this naive way sometimes will produce incorrectly rounded values. OP said this is important. Well, maybe — ML is a magic for me But with conversion from Python floats to float32 we already lose a lot of precision bits…
I think we must be talking past each other, each repeating things the other already knows, but failing to locate where the disagreement (if any) is.
I’m saying there’s NO NEED for CPython to have ANY code to convert between 16- and 32-bit float types. Those aren’t types to Python, just STORAGE FORMATS.
We have code already to do the 'hard parts" (rounding 64-bit floats back to the IEEE 32- and 16-bit formats). Build on that - it should be all but trivial to copy/paste/edit the existing code to round back 64-bit floats to 32-bit floats, modifying it to round back to a 16-bit float instead. The sign and exponent bits are identical in IEEE single and bfloat16 formats.
We already have code to round. Reuse it, just change the bit location to which it rounds. I have no trouble believing good rounding is crucial in this context; indeed, the fewer the bits of precision (and bfloat16 has very few), the more important it is to preserve all the accuracy possible.
From the little I’ve read, Intel’s bfloat16 HW arithmetic will actually take place in 32-bit registers and use 32-bit float circuits, but sped a little by knowing in advance that the last 16 bits of inputs are always 0. Sounds like they’re begging for “double rounding” glitches in the end (to convert 32-bit results back to 16 for storage), but don’t know.
And, of course, Python doesn’t need to know anything about that
(Nowadays we have only code for conversion to/from half-floats. The rest goes just by copying bits & type conversions, in Pack4/Unpack4.)
Yes, it’s easy to adapt PyFloat_Pack2/Unpack2 for bfloat16 format, but I’m not about implementation of the OP proposal (i.e. conversion to/from bfloat16 storage format of CPython’s floats). The point is that his wishes could be at least partially accomplished, using existing capabilities of the struct module, i.e. by support of the float32 storage format. Just pad/shrink such value. No changes needed in the CPython at all.
This certainly fits my intuition about ordinary use cases for floating-point arithmetic, but not ML.
Of course, I trust OP that incorrect rounding, that may be happened in this way for conversion to the bfloat16 is inappropriate for ML applications.
Sorry, I didn’t explain my concern very well. When NumPy fills in a buffer (as in the Python buffer protocol) it is setting the format to "Zf” to indicate an array of complex single precision numbers. (The first letter is Z not two.) It appears they’ve implemented a rejected section of PEP 3118.
When I use PyObject_GetBuffer(), I see that view->formatis the two-letter string "Zf” for a NumPy array having dtype np.complex64. I think they are obligated to use the format character "F” as documented in the Python docs for struct. If not, I’m confused as to how the protocol works. Maybe it should be documented that consumers are advised to accept either format string.
Yes, it should be very straightforward to reuse the existing CPython rounding code. I see no technical problems whatsoever.
Intel’s (and AMD’s) circuitry will work fine. Note that 32 bfloat16 elements are packed in a 512-bit SIMD register, so while there’s sure to be transistors that are re-used between different floating-point circuits, it’s misleading to imagine that float32 circuitry is reused simply by zero-padding each bfloat16 to a float32. For one thing, it would be expensive to expand 512 bits to 1024….
I can’t say whether numpy’s use of type codes is “a bug”, because I couldn’t find a definitive account of what “the rules” are. Closest I got:
“struct module style syntax” doesn’t spell out the allowable values, and it’s impossible for me to guess whether “style” was used deliberately to abstract away from the specific character codes struct uses.
I think this is better addressed in a numpy forum. My preference, of course, is to use the same codes as struct uses whenever possible. But we can’t tell numpy what to do regardless.
That makes more sense. I was going from “industry observer accounts” that said Intel would take different approaches, over time, in different generations and variations of x86-64.
Type Object Structures — Python 3.14.3 documentation goes over how one should actually interact with this, and the very first step is that you have to confirm you can actually handle the request. Not having knowledge of the format string is a reason to reject it.
Nothing in the buffer protocol requires that all libraries speak the same format.
.