Where to post this unexpected feature

Python Help
1

Hello everybody,

I came across something that was quite unexpected in my eyes.

    # Code running on Windows 10, Python version 3.11.6
    import os
    import shutil
    directory = r'W:\prod'
    src = r"W:\test\fill_icon.png"
    for file in os.scandir(directory):
      dest = os.path.join(r"W:\test", file)
      shutil.copy(src, dest)

I was thinking this small piece of code would copy the prod file over to the test location containing the fill_icon.png content. To my suprise I instead overwrote the production data with the fill_icon.png. I don’t understand how this is correct behaviour of the code. Can anyone take a look into this and see if should instead return an error when joining a dir entry this way?

2

Scandir returns absolute paths not the filenames.
os.join.path will always return the last absolute path, ignoring earlier paths.
os.path.join('/x/y', '/a/b/d') returns /a/b/d not /x/y/d.

In other words your code worked as expected.

I suspect you you wanted to do is os.path.join(r"w:\test", os.path.basename(file))

When I’m writing code like this I first run it printing out the src and dest to check my logic is as I expect.

3 Likes
3

I have noticed this before, but in a slightly different context. Would it be possible to explain why it is logical that os.path.join('/x/y', '/a/b/d') == /a/b/d?

The context where it ‘confused’ me is Path("/a/b/c/") / "/d/e/f/" == Path('/d/e/f').

4

Fyi the docs are her os.path — Common pathname manipulations — Python 3.8.20 documentation

This is my guess at why it works this way.
It works this way because there are a number of possible solutions when an absolute path is in the middle of the args.

  1. Start the join again, as os.path.join does
  2. Raise an exception
  3. Something else, which is likely to be some form of guess as to what the user intended

(1) has proved to be workable in practice. As it allows a default path to be overridden by a user input absolute path. But if only a basename is used then the default path is joined on to the basename.

(2) requires a lot more logic in the caller

(3) will likely lead to support/maintenance issues as there is no obvious correct behaviour.

1 Like
5

In that case the aspect that I was missing is that a path that starts with ‘/’ should be interpreted as an absolute path.

6

It would be more obvious on Windows that C:/... cannot be prefixed.

7

But C:name can. The C: does not make it absolute on Windows.
You still need either / or \ after the device to make it absolute.

Both \name and C:\name are considered absolute on Windows.
But name and C:name are not.

8

The thing that makes this one a bit more confusing, is that the representation of os.scandir object only prints «
<DirEntry ‘filename.ext’>» so its easy to think its just the filename on not an absolute path element.

ons. 2. okt. 2024 kl. 12:18 skrev Barry Scott via Discussions on Python.org <notifications@python1.discoursemail.com>:

9

I was also confused by seeing only the name.
I had to convert to a path by access the .path member of DirEntry.
You can also use the os.fspath() function to get a file system path from an object that contains one.