IOError: [Errno 2] No such file - Paramiko put()

Question

I'm uploading a file via SFTP using Paramiko with sftp.put(localFile, remoteFile). I make the necessary directory first if needed with

    makeCommand = 'mkdir -p "' + remotePath + '"'
    ssh.exec_command(makeCommand)

this was works sometimes but I'm occasionally getting the following error:

    sftp.put(localFile, remoteFile)
    File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 565, in put
    File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 245, in open
    File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 635, in _request
    File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 682, in _read_response
    File "build/bdist.macosx-10.8-intel/egg/paramiko/sftp_client.py", line 708, in _convert_status
    IOError: [Errno 2] No such file 

despite the fact that the local file definitely exists (and localFile is the correct path to it) and the remote path is made. There is discussion here and here on a similar problem but none of the points raised there have helped me. My server supports the df -hi command. Has anyone any advice on this or a possible solution?

EDIT

After suggestions below I tried changing the working directory with sftp.chdir(remoteDirectory) but this call produced the exact same error as above. So it seems this isn't just an upload issue. Any ideas?

Solution 1

It seems to be a remote folder permission problem. Although the remote folder was made before the file was uploaded, it appears the permissions on the folder were preventing an upload.

The problem is linked to this issue - if I set open permissions on the folder I'll be uploading to before I upload, the program can upload fine. Although for a permission issue I should be getting IOError: [Errno 13] Permission denied, since I made the changes I haven't encountered any errors.

I'm not sure if it's the response the server is giving Paramiko which is the issue, or a bug in Paramiko itself which is causing IOError: [Errno 2] No such file instead of a Errno 13, but this appears to have solved the problem.

Solution 2

The put method has a confirm parameter which is enabled by default, which will do a stat on the file after transfer.

In my case, the remote server i was transferring the file to, immediately moved any transferred files to another location to get processed which was causing the stat to fail. Setting the confirm parameter to False resolved this.

def put(self, localpath, remotepath, callback=None, confirm=True):

From the paramiko source sftp_client.py:

:param bool confirm: whether to do a stat() on the file afterwards to confirm the file size (since 1.7.7)

Solution 3

The IOError is local, so (for whatever reason) it seems that your local python cannot find localFile. Safety checking this before the call might help tracking down the problem:

if os.path.isfile(localFile):
    sftp.put(localFile, remoteFile)
else:
    raise IOError('Could not find localFile %s !!' % localFile)

If you're positive that localFile does exist, then this could just be a path problem - is localFile on an absolute or relative path? Either way, the if statement above will catch it.

EDIT

Tracing through the paramiko files shows that line 245 of sftp_client.py (the one throwing the exception) is actually

fr = self.file(remotepath, 'wb')

which is quite misleading as paramiko throws an IOError for a remote file! My best guess now is that remoteFile is either a missing directory or a directory you don't have access to.

Out of interest, can you list the remote dir

sftp.listdir(path=os.path.dirname(remoteFile))

to check that it's there (or maybe it's there and you can write to it)?

Solution 4

Are you sure the directory has been created and it is your remote working directory? Paramiko has its own methods for creating new directories and navigating the remote file system. Consider using something like:

sftp.mkdir(remotedirectory)
sftp.chdir(remotedirectory)
sftp.put(localfile, remotefile)

Solution 5

I faced the same issue. it was a silly mistake.

• Just use sftp.stat(your remote directory) to check if it's there.
• then use sftp.put(localfileabsolutepath, remotedir+filename)

It will work for sure.

View more solutions

Comments

[Cian]

Thanks, go to know it's local. Strangely, I tried opening the file just before uploading, just checking it could be opened, and now it's uploading fine. So if basically have just open it to see first then close it and upload and for some reason that is working now. No idea why. Will add the check like you suggested for safety until I work out the cause!

[Cian]

Scratch that, it was a coincidence it worked. Uploads were working for about an hour, and now I'm getting the same error. Even if I check the file exists and can open the file, I get the above error when trying to upload?

[Cian]

Hi, thanks for the edit. Ya I can list the remote directory, and I ca access it. I had a guess that maybe Paramiko was expecting a file to be present at the remote location? So I'm trying a 'touch <remoteFile>' using ssh.exec_command() to make a blank file first. Strangely my app will upload fine for some time, then randomly stop working with the error again.

[Cian]

The remote file is definitely made, I'm certain. For a file put/get it isn't necessary to chdir if you pass to whole path to the file. Uploads are working at the moment, since I introduced a retry and touch, but as a backup, to chdir and then put the file could be an option. Could implement this as a fallback if the error arises from the first attempt.

[Cian]

I tried this as you suggested, and strangely sftp.chdir() produces the same error. See my edit above.

[Brian]

Can you confirm that the directory you are attempting to write to or chdir into already exists. Also, check to see if there is anything in syslog that may be of use.

[Cian]

Yes, certain it does. As I mentioned above I can even use touch to create a file at that location.

[Cian]

Strangely, even sftp.listdir() is throwing the same error now, even if I make that directory first to be sure it exists?

[Spacez]

Oh my god. This is definitely what I was looking for! My remote server immediately moves any transferred files to another location, and I was facing the same problem. Thank you so much!