Hello Guys/Gals…I have script to copy files from folder to another folder. But when the folders are located to server, it will copy only 1 file and got error.
My code is:
import os
import shutil
import pandas as pd
from pathlib import Path
source_folder = r"\\asia1212.intra.company.com\user\name\folder\\"
destination_folder = r"\\ishare.td.company.com\sites\Intranet\SharedDocuments\2023\Review\\"
for root, dirs, files in os.walk(source_folder):
for file in files:
src_file_path = os.path.join(root, file)
dst_file_path = os.path.join(destination_folder, file)
shutil.copy(src_file_path, dst_file_path)
The error is below
FileNotFoundError: [WinError 2] The system cannot find the file specified: '\\\\ishare.td.company.com\\sites\\_Intranet\\SharedDocuments\\2023\\Review\\\\P001.doc'
But when i check the destination folder, i saw P001.doc is there.
Please guide me…help me fix this…I want to copy all files…thanks
I don’t use any MS Windows systems, so I can’t test this for you, but my feeling is that you should not be using raw strings; rather use the path objects that the Pathlib classes provide.
I have learned a good deal from this tutorial:
… and re-coded my file copying app based on what I learned from that.
As rob suggested pathlib is nice to use, i also recommend it
But i do not see an error that pathlib will fix with your code.
I suggest that you print both source and destination paths before the copy.
Then when it fails you can cut-n-paste the paths and test them using dir and copy from a cmd.exe window.
Barry. You’re more experienced than me, so maybe you don’t see an issue with:
… but to me, that path seems to be a little off and I suspect that it’s because of the incorrect use of a string/.join() method? Or is it because of the server connection?
And would it be not better to use the ftplib for this kind of a project?
… has just popped up. Although it’s aimed at files on a local host, rather than server → server, it does explain The Problem With Representing Paths as Strings and such may be of some help to you.
With repr adding lots of \ its hard to see the path.
Also we cannot test as we are not on the OP network.
That is why i suggested print the paths so that can be tested with windows dir and windows copy commands.
FTP should not be used on anything modern, its a security nightmare.
Fair point, but this is (from what I can see) intended for use on an internal network. If that’s still an issue, there’s the .FTP_TLS subclass. I only suggest this as it [server → server file coping] is a feature that I’m looking to add to my own file copying/backup app.
I notice that the path shown in the error message contains \\_Intranet\\, whereas the destination path in the script contains \Intranet\ in the raw string, so are you sure that the P001.doc you see is the same as the one it says it can’t find?
The problem might be that os.walk is walking over the source hierarchy, down into the source folder tree, but you’re copying only the files and not creating their parent folders on the destination server.
Using SMB mounted file systems is far better the any of the FTP variants.
After that you would want to look at SSH’s SFTP (nothing to do with FTP)
and rsync.
You need to provide the complete traceback, not just the exception. It’s necessary to know whether copy() is failing in the internal copyfile() call when it tries to open() the destination file, or the internal copymode() call when it calls chmod() on the destination file.
Getting ERROR_FILE_NOT_FOUND (2) instead of ERROR_PATH_NOT_FOUND (3) means that “'\\ishare.td.company.com\sites\_Intranet\SharedDocuments\2023\Review” probably exists and is accessible, and the likely problem was that “P001.doc” didn’t exist yet. That seems like a failure from calling chmod() in copymode().
It could be that “P001.doc” is in the SMB client’s file-not-found cache because copyfile() tries to stat() the destination file before opening it for writing. If you keep experiencing a similar error, try setting the “FileNotFoundCacheLifetime” value to 0 on the client machine. If that doesn’t help, try setting the “DirectoryCacheLifetime” value to 0.
If you don’t feel comfortable setting the SMB “FileNotFoundCacheLifetime” (5 second default) and “DirectoryCacheLifetime” (10 second default) registry values to 0, as I previously suggested, then try using shutil.copyfile() instead of shutil.copy(). Using just copyfile() is fine because the copymode() call in shutil.copy() is pretty much worthless on Windows[1].
Python should not implement POSIX os.chmod() on Windows. The implementation uses the readonly file attribute as if it’s the logical inverse of POSIX write permission, but the behavior is very different from what’s specified by POSIX. Removing write permission from a file should not prevent deleting the file, while removing write permission from a directory should prevent deleting files and directories linked in the directory. The hack using the readonly file attribute on Windows fails on both accounts.
The readonly attribute prevents modifying or deleting the data streams in a file or directory (e.g. named NTFS $DATA streams in a file or directory). It’s a bit like the immutable attribute in Linux, macOS, and BSD, but it’s more limited. It doesn’t protect metadata (e.g. time stamps, attributes, extended attributes, reparse point, security descriptor) or the index stream in a directory (i.e. the files and directories that are linked in a directory). To modify the readonly attribute, there should be an os.set_file_attributes() function that corresponds to the st_file_attributes field of the os.stat() result on Windows. ↩︎