How to convert LP_c_ubyte to PIL buffer

While trying to get GetAchievementIcon() to work in Ren’Py using its steamapi I’m running into difficulty getting GetImageRGBA() to use a buffer which can then be passed to PIL.Image.frombuffer() to save it as a PNG file.

I’m basically trying to get a Python implementation of this example from the Steamworks documentation:

HGAMETEXTURE GetSteamImageAsTexture( int iImage )
{
	HGAMETEXTURE hTexture = 0;

	// You should first check if you have already cached this image using something like a dictionary/map
	// with iImage as the key and then return the texture handle associated with it if it exists.
	// If it doesn't exist, continue on, and add the texture to the map before returning at the end of the function.

	// If we have to check the size of the image.
	uint32 uAvatarWidth, uAvatarHeight;
	bool success = SteamUtils()->GetImageSize( iImage, &uAvatarWidth, &uAvatarHeight );
	if ( !success ) {
		// Log a warning message.
		return hTexture;
	}

	// Get the actual raw RGBA data from Steam and turn it into a texture in our game engine
	const int uImageSizeInBytes = uAvatarWidth * uAvatarHeight * 4;
	uint8 *pAvatarRGBA = new uint8[uImageSizeInBytes];
	success = SteamUtils()->GetImageRGBA( iImage, pAvatarRGBA, uImageSizeInBytes );
	if( !success )
	{
		// Do something to convert the RGBA texture into your internal texture format for displaying.
		// hTexture = m_pGameEngine->HCreateTexture( pAvatarRGBA, uAvatarWidth, uAvatarHeight );
		// And add the texture to the cache
	}

	// Don't forget to free the memory!
	delete[] pAvatarRGBA;

	return hTexture;
}

My code starts by registering a handler for the UserAchievementIconFetched_t callback:

init python
    def achievement_icon(cb):
        ...

    _renpysteam.callback_handlers["UserAchievementIconFetched_t"].append(achievement_icon)

achievement_icon() receives a callback object like this one:

cb: <steamapi.UserAchievementIconFetched_t object at 0x7fb0fda2f140>
cb.m_nGameID: 210970
cb.m_rgchAchievementName: <steamapi.c_byte_Array_128 object at 0x7fb0fda2f340>
cb.m_bAchieved: True
cb.m_nIconHandle: 15

cb.m_nIconHandle is passed to GetImageSize() to get the width and height:

        icon_width = c_uint32(0)
        icon_height = c_uint32(0)

        success = achievement.steamapi.SteamUtils().GetImageSize(
            cb.m_nIconHandle,
            byref(icon_width),
            byref(icon_height),
        )

This works as expected:

icon_width: c_uint(64)
icon_height: c_uint(64)
icon_width.value: 64
icon_height.value: 64

We then need to create the RGBA buffer and pass it to GetImageRGBA():

        icon_size = icon_width.value * icon_height.value * 4
        icon_rgba = (c_uint8 * icon_size)()

        success = achievement.steamapi.SteamUtils().GetImageRGBA(
            cb.m_nIconHandle,
            byref(icon_rgba),
            icon_size,
        )

This fails with a TypeError:

  File "game/script.rpy", line 70, in achievement_icon
    success = achievement.steamapi.SteamUtils().GetImageRGBA(
  File "lib/python3.9/steamapi.py", line 5361, in GetImageRGBA
ArgumentError: argument 3: <class 'TypeError'>: expected LP_c_ubyte instance instead of pointer to LP_c_ubyte_Array_16384

Casting to POINTER(c_ubyte) resolves the error:

        icon_rgba = cast((c_uint8 * icon_size)(), POINTER(c_ubyte))

        success = achievement.steamapi.SteamUtils().GetImageRGBA(
            cb.m_nIconHandle,
            icon_rgba,
            icon_size,
        )

        data = bytes(icon_rgba.contents)

        img = Image.frombuffer(
            "RGBA",
            (icon_width.value, icon_height.value),
            data,
            "raw",
            "RBGA",
            0,
            1,
        )
        img.save("images/icon.png")

But this fails with a ValueError:

  File "game/script.rpy", line 93, in achievement_icon
    img = Image.frombuffer(
  File "python-packages/PIL/Image.py", line 3037, in frombuffer
  File "python-packages/PIL/Image.py", line 2979, in frombytes
  File "python-packages/PIL/Image.py", line 809, in frombytes
  File "python-packages/PIL/Image.py", line 397, in _getdecoder
ValueError: unknown raw mode for given image mode

I think this is because the content of the buffer is only the first byte, likely because we’re not casting to an array:

icon_rgba: <steamapi.LP_c_ubyte object at 0x7f4020fa9c40>
icon_rgba.contents: c_ubyte(34)
data: b'"'

How can I pass a buffer to GetImageRGBA() which can then loaded by PIL?

I don’t think the cast was necessary here. The more significant change is that the broken code used byref and the working version doesn’t. Think carefully about how pointer decay works in C.

Well, that would be a problem, since you created a separate buffer (via the bytes call). But it’s clearly not the problem that the error message complains about. Read it carefully: “unknown raw mode for given image mode”. Then look at the arguments used for the frombuffer call, and match them up with the documentation, cross-referencing with the documentation for the “raw” decoder:

Doesn’t the “raw_mode” value look suspicious? Doesn’t that resemble something said in the error message?

To answer the question, as far as I’m aware, Array types created by ctypes already support the buffer protocol. The point of using them is that Python is managing the memory. You should be able to pass the (c_uint8 * icon_size)() directly, and also use it directly in Image.frombuffer.

Thanks for spotting that issue in my code, this works as expected:

        icon_rgba = (c_uint8 * icon_size)()

        success = achievement.steamapi.SteamUtils().GetImageRGBA(
            cb.m_nIconHandle,
            icon_rgba,
            icon_size,
        )

Indeed, by passing the buffer pointer to GetImageRGBA() correctly, it can then also be loaded successfully by PIL without converting to bytes first.

Yet another issue spotted by an extra pair of eyeballs. Thank you, again.

Fixing the “RBGA” typo also makes this work as expected:

        img = Image.frombuffer(
            "RGBA",
            (icon_width.value, icon_height.value),
            icon_rgba,
            "raw",
            "RGBA",
            0,
            1,
        )
        img.save(icon_file)

This is indeed the solution.

If I understand correctly, since we used ctypes to allocate the memory for the buffers we don’t have to explicitly free it like in the Steamwork example:

	// Don't forget to free the memory!
	delete[] pAvatarRGBA;
1 Like

Looks good. And yes, there is no explicit “free” built in to ctypes, for that exact reason: it’s either using memory that the C code is entirely responsible for, or memory that is being managed by the Python virtual machine.

Of course, there do exist some more questionable C APIs that allocate memory internally and give you a pointer to it, with the expectation that you’ll read or modify it, but the API is responsible for cleanup when you call some other API function. In these cases you’ll need to do a bit more work, and be really sure of the ownership and lifetime semantics.

But in your case it’s very simple. ctypes created an object with internal buffer storage, and passed a pointer to that storage to the API. The API naively treated that memory like any other pointed-at memory, as one does in C, and returned a success code (you should probably check that value and raise an exception on failure, btw). The object which holds the buffer is still an ordinary Python object, and (assuming the C code didn’t have a buffer overrun) is still intact. The memory allocation will be cleaned up by the Python GC, just as it would be for any other object.

You can, of course, del references to that object, but that isn’t the same thing as freeing memory - since the object can only be GCd when there are no remaining references.