How to fix Windows 8 Anaconda "can't open file" errors in cygwin when using absolute paths?

Question

I'm having troubles getting Anaconda to run python scripts on cygwin.

My configuration. Windows 8.1

$ echo $PATH
/usr/local/bin:/usr/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/WINDOWS/Syste
m32/WindowsPowerShell/v1.0:/cygdrive/c/Anaconda:/cygdrive/c/Anaconda/Scripts

launching python or ipython with no script works fine:

$ which python
/cygdrive/c/Anaconda/python

$ python
Python 2.7.6 |Anaconda 1.9.2 (64-bit)| (default, Nov 11 2013, 10:49:15) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()

$ ipython
Python 2.7.6 |Anaconda 1.9.2 (64-bit)| (default, Nov 11 2013, 10:49:15) [MSC v.1500 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]:
Do you really want to exit ([y]/n)? y

$ echo $PATH
/usr/local/bin:/usr/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/WINDOWS/Syste
m32/WindowsPowerShell/v1.0:/cygdrive/c/Anaconda:/cygdrive/c/Anaconda/Scripts

Here's the problem:

$ pwd
/home/John

$ ls -l /cygdrive/c/cygwin64/home/John/FooDir/helloworld.py
-rwxrwx--x+ 1 John None 47 Apr 25 16:42 /cygdrive/c/cygwin64/home/John/FooDir/helloworld.py

$ cat /home/John/FooDir/helloworld.py
#!/usr/bin/env python
print("Hello, World!")

$ python FooDir/helloworld.py
Hello, World!

$ python /home/John/FooDir/helloworld.py
C:\Anaconda\python.exe: can't open file '/home/John/FooDir/helloworld.py': [Errno 2] No such file or directory

$ python /cygdrive/c/cygwin64/home/John/FooDir/helloworld.py
C:\Anaconda\python.exe: can't open file '/cygdrive/c/cygwin64/home/John/FooDir/helloworld.py': [Errno 2] No such file or directory

$ python C:\\Cygwin64\\home\\John\\FooDir\\helloworld.py
Hello, World!

This happens with any script that I try to run with absolute paths. I suspect the problem is caused by some sort of cygpath issue, but don't know how to fix it...

I don't have this problem when using Cygwin's version of python.

Solution

Cygwin performs a mapping between Windows paths and the paths that Cygwin programs see. For example, your Cygwin HOME directory /home/John is the Windows directory C:\cygwin\home\John. Putting the root of the Cygwin file system at the Windows C:\cygwin directory avoid collisions between Cygwin root directory names and Windows root directory names.

Cygwin also maps the root directories of Windows drives to directories under /cygdrive, so that Windows directories C:\ and D:\ are equivalent to Cygwin directories /cygdrive/c and /cygdrive/d.

Windows programs don't understand Cygwin absolute paths and Cygwin programs (generally) don't understand Windows absolute paths. You can convert between the two forms using the cygpath command. cygpath --help gives a pretty complete description of its capabilities.

Your Windows python program understands only Windows paths, so to get it to execute python /home/John/FooDir/helloworld.py, use cygpath like this:

python $(cygpath -w /home/John/FooDir/helloworld.py)

Or, knowing what you now know about the mapping between Cygwin paths and Windows paths, you could just run your Python script like this:

python "C:\cygwin\home\John\FooDir\helloworld.py"

Comments

[John Prior]

The use of Windows paths allowed me to launch the scripts, but now I'm having trouble setting $PYTHONPATH correctly so that my custom modules can be imported into those scripts. I've tried using both Windows and Cygwin absolute paths in $PYTHONPATH but I still get "ImportError: No module named ..." errors.

[garyjohn]

One thing you might try, if you haven't already, is executing export PYTHONPATH before running your script. Another is to execute python to run the interpreter, then within the interpreter execute import sys and sys.path. By looking at the value of sys.path you might be able to see the problem with your setting of PYTHONPATH.

[John Prior]

I finally figured out my problem: I was using cygwin symbolic links in my paths; and so even though the paths looked "correct" in sys.path, they wouldn't work. Only absolute, non-symbolic paths seem to work.