Shutil.copyfile Corrupts Copied File When Running Under pythonw on Windows 10 Home

Python Help
1

I encountered an issue where shutil.copyfile corrupts the copied file when executed under pythonw.exe on Windows 10 Home. However, running the same operation with python.exe works correctly.

Issue Description

  • Problem: When using shutil.copyfile in a .pyw script executed with pythonw.exe, the copied file becomes corrupted.
  • Observation: Running the same .pyw script with python.exe results in a correctly copied file.

Workaround

  • Solution: Execute the shutil.copyfile operation using python.exe instead of pythonw.exe.

Test Case

To reproduce the issue, use the following script:

program.pyw

import os, subprocess, shutil, sys, tempfile
os.makedirs(os.path.join(tempfile.gettempdir(), "modifield_python"), exist_ok=True)
renamed_pythonexefile = shutil.copyfile(sys.executable, os.path.join(tempfile.gettempdir(), "modifield_python\\modified_python.exe"))

For demonstration, this script creates a temporary directory named modifield_python and copies the current Python.exe executable, renaming to modified_python.exe.

Testing:

Upon launching the by double clicking C:\Users\%USERNAME%\AppData\Local\Temp\modifield_python\modified_python.exe, you will sense that something is not right already: it does not open. The size of the copied executable seems reasonable, but it just doesn’t work anymore.

2

I suggest you compare the contents of the two files.
I expect you will find they are identical.

The issue is likely to be that a windows gui .exe needs additional meta data files for it to launch, which you did not copy.

1 Like
3

Yeah I suspected it might be the case. However they are not completely identical as you would expect.

image
image841Ă—546 64.5 KB

Later on I will try to use shutil.copy2 and see if it copied file will be functional.

To preserve all file metadata from the original, use copy2() instead.

4

They are not identical, so no, it does not seem reasonable.

1 Like
5

shutil.copy2() worked.

By simply replacing shutil.copyfile() with shutil.copy2() it all started to work.
Mostly likely it was due to metadata not being copied.

The size of copied file might not be as original, but it’s still identical to shutil.copyfile()

image
image543Ă—674 16.7 KB

The previously mentioned working example with shutil.copy2()

import os, subprocess, shutil, sys, tempfile
os.makedirs(os.path.join(tempfile.gettempdir(), "modifield_python"), exist_ok=True)
renamed_pythonexefile = shutil.copy2(sys.executable, os.path.join(tempfile.gettempdir(), "modifield_python\\modified_python.exe"))

This script should produce modified_python.exe that upon double clicking it, will open python console.

Path where the produced file resides: C:\Users\%USERNAME%\AppData\Local\Temp\modifield_python\modified_python.exe

For me, it seems the issue is solved.

6

copyfile() calls _copyfileobj_readinto() on Windows:

Is it possible that readinto() can read n < length bytes without reaching the end of the file? If so I think the break on line 192 might be at fault.

1 Like
7

This ends up at a single call to _Py_read from what I can tell, for which the docstring only states that EOF can be assumed when the return value is 0.

On success, return the number of read bytes, it can be lower than count.
  If the current file offset is at or past the end of file, no bytes are read,
  and read() returns zero.

So yes, from what I can tell it’s invalid to assume that readinto returning less than length bytes means we are at an EOF since C doesn’t guarantee that behavior for the builtin read, and python doesn’t do anything to guarantee that behavior instead.

1 Like
8

According to your info.
You are comparing python.exe to a copy of pythonw.exe they will not be the same.

2 Likes
9

Well, it seems like the mystery is solved. I’ve compared against pythonw, and yeah, they are identical. This is overlook on my part as I’ve overfocused on the metadata, not realizing that I’m using statement sys.executable which becomes either pythonw.exe or python.exe and they are different in size.

10

Note that executing “modified_python.exe”, as copied by your example, requires Python’s installation directory to be in the PATH environment variable, in order to find the dependent DLLs “python312.dll” and “vcruntime140.dll”. Normally this is not required because they’re located in the directory that contains the executable – “python.exe” or “pythonw.exe”, which the system searches implicitly. To avoid the extra PATH requirement, you’ll have to symlink, hardlink, or copy those two DLLs to the temp directory that contains “modified_python.exe”.

2 Likes
11

Here is the utiilty script.

usage: windows_create_renamed_copy_of_python_executable.py [-h] file_path

Produce renamed Python Executable on Windows Operating System.

positional arguments:
  file_path   Path to the new executable file

options:
  -h, --help  show this help message and exit

Example usage: python windows_create_renamed_copy_of_python_executable.py ".\test.exe"

windows_create_renamed_copy_of_python_executable.py

def produce_renamed_python_executable(new_executable_path: str):
    import os, shutil, pathlib, sys
    python_executable_installed = pathlib.Path(sys.exec_prefix, 'python.exe')  # change to pythonw to create a pythonw executable.
    python_executable_new = pathlib.Path(new_executable_path)
    os.makedirs(os.path.dirname(new_executable_path), exist_ok=True)
    return str(os.path.abspath(shutil.copy2(python_executable_installed, python_executable_new)))

def main():
    import argparse
    parser = argparse.ArgumentParser(
        description='Produce renamed Python Executable on Windows Operating System.', 
        epilog='Example usage: python %(prog)s ".\\test.exe"')
    parser.add_argument('file_path', type=str, help='Path to the new executable file')
    try:
        new_executable_path = produce_renamed_python_executable(parser.parse_args().file_path)
        print(f"Successfully created new executable at: {new_executable_path}")
        return 0
    except FileNotFoundError:
        print("Error: Source Python executable not found.", file=sys.stderr)
        return 1
    except PermissionError:
        print("Error: Permission denied. Make sure you have the necessary rights.", file=sys.stderr)
        return 2
    except Exception as e:
        print(f"Error: An unexpected error occurred: {str(e)}", file=sys.stderr)
        return 3

if __name__=="__main__": 
    import sys
    sys.exit(main())