Support for os.sendfile for Windows

Currently, os module for windows does not have support for sendfile. However, it exists for Windows family of operating systems, but only for sockets.

I think it would be great to close one of the gaps between POSIX operating systems and Windows, at least for that. Here’s windows socket call called TransferFile, which should work similarly as os.sendfile is.
TransmitFile at learn.microsoft.com

Maybe open an issue on the CPython issue tracker?

If you see this valuable, then sure. When doing the issue on GitHub, it deliberately asks if this was first discussed in ideas/discuss page outside GitHub, so I did that :slightly_smiling_face:

1 Like

Fair enough—just to be clear, I have no particular opinion, nor am I particularly qualified on this topic. It just seemed to me that adding missing support for a Tier 1 platform to an existing standard library function (as opposed to, e.g., introducing a whole new function) seemed far enough from a brand new feature driven by conceptual and usability concerns, and close enough to a detail enhancement/defect fix driven by technical and implementation ones, to be a better fit for the issue tracker. However, it’s perfectly possible I’m misjudging this.

1 Like

Then I will create issue on GitHub as soon as possible :slight_smile:
It’s not the only one missing, other things like usage of WSAAsyncSelect for select.select function etc.

IMO, I’m all for reducing the delta between what’s implemented on POSIX platforms vs. on Windows (so long as there aren’t subtle but significant differences we can’t account for that could bite unwary users), so great to see you’re looking in to that. Thanks!

1 Like

TransmitFile() is supported by the asyncio proactor event loop sendfile() and sock_sendfile() methods. Here’s a snippet of the low-level implementation in “Modules/overlapped.c”:


WSAAsyncSelect() isn’t applicable to implementing select.select(), and it isn’t a good fit for the standard library. It integrates socket I/O into a GUI window message loop. That’s for a GUI toolkit to handle, such as Qt for example:

OTOH, supporting WSAEventSelect() and WSAEnumNetworkEvents() would be useful for integrating socket I/O with WaitForMultipleObjects() waits.

Thank you a lot for this insight! I didn’t know that WSAAsyncSelect is for GUI window message loop and that TransmitFile is already in!
Starting from which Python version it’s usable for everybody? And additionally - can we use it without asyncio? :slightly_smiling_face:

Later on I will take a look at other things, like more socket related ioctl methods available on Windows, but not exposed to python :slight_smile:
Thanks again!

Seen in a vacuum, “only works on sockets” is a pretty significant and annoying difference. But that exact same distinction also applies to quite a few other Windows features (where winsock is its own thing, in contrast to Unix where those features are available on all FDs), making it less confusing.

+1. Definitely worth adding.

1 Like

The Overlapped.TransmitFile() method is implemented in Python 3.7+ in the _overlapped extension module. It’s undocumented and only intended for internal use in the standard library. That said, it’s technically possible to use it outside of the asyncio module, but it’s awkward. For example:

>>> import socket, msvcrt, _overlapped
>>> f = open('spam.txt', 'wb+')
>>> f.write(b'eggs')
4
>>> f.flush(); f.seek(0)
0
>>> h = msvcrt.get_osfhandle(f.fileno())
>>> sr, sw = socket.socketpair()
>>> ov = _overlapped.Overlapped()
>>> ov.TransmitFile(sw.fileno(), h, 0, 0, 4, 4, 0)
>>> ov.getresult(True)
4
>>> sr.recv(4)
b'eggs'

Thanks a lot! I have a question then - if most of the work was done to support this, why it’s not hooked onto os.sendfile? That would make life much easier in terms of code portability etc.
If there’s issue with time needed for somebody to implement it, maybe I might jump in (if it does not involve C too much, my skills are not great :smiley: ) and implement that for os.sendfile in my rather limited, but still available, spare time :slight_smile: