How does Linux handle multiple consecutive path separators (/home////username///file)?

Question

I'm working on a python script that passes file locations to an scp subprocess. That's all fine, but I'm in a situation where I may end up concatenating a path with a filename such that there's a double '/ in the path. I know that bash doesn't care if you have multiple file separators, but I'm wondering how exactly that is rectified. Is it bash that strips extra /s or does it really not matter ever?

I ask because it will save me several lines of code to check for extra /s while concatenating. I know it's not a big deal, but I'm curious as well. I have a bash script that has the line cd //usr (instead of cd /usr), which seems to imply there might be a significance to using multiple /s in a path

Solution 1

Multiple slashes are allowed and are equivalent to a single slash. From the Single Unix specification (version 4), base definitions §3.271 pathname: “Multiple successive slashes are considered to be the same as one slash.”

There is one exception: If a pathname begins with two successive slash characters, the first component following the leading slash characters may be interpreted in an implementation-defined manner. (ref: base definitions §4.13 pathname resolution). Linux itself doesn't do this, though some applications might, and other unix-ish system do (e.g. Cygwin).

A trailing / at the end of a pathname forces the pathname to refer to a directory. In (POSIX 1003.1-2001 (Single Unix v4) base definitions §4.11 pathname resolution, a trailing / is equivalent to a trailing /.. POSIX 1003.1-2008 (Single Unix v4) base definitions §4.13 removes the requirement to make it equivalent to /., in order to cope with non-existing directories (e.g. mkdir foo/ is required to work, whereas mkdir foo/. wouldn't — see the rationale for the change).

For programs that act on a directory entry, if foo is a symbolic link to a directory, then passing foo/ is a way to make the program act on the directory instead of the symbolic link.

¹ _{Note that this applies for pathname resolution only, i.e. when accessing files. Filename manipulations may work differently. For example basename and dirname ignore trailing slashes.}

Solution 2

The OS doesn't appear to care about it either, having just tried out a C program with a direct syscall to open with a // in the path.

You can use the python library function os.path.normpath to normalize it though, which saves you having to scan through the string looking for extras. Other languages have similar functions.

http://docs.python.org/library/os.path.html#os.path.normpath

Solution 3

On all Unix systems that I've seen it's the same as a single /, but the Unix standard specifies that

A pathname that begins with two successive slashes may be interpreted in an implementation-defined manner, although more than two leading slashes shall be treated as a single slash.

so it may be handled specially, depending on your system. (Some older Unix versions used a double leading / for remote filesystem access, and there may still be some that do.)

Solution 4

Use os.path.join in Python and you won't get multiple slashes. Building up filenames yourself by concatenating strings is considered poor Python style.

Solution 5

There is no difference.

Multiple slashes get ignored (without effect), e.g.:

ls -al //usr///////bin/sed

View more solutions

Comments

[Stefan]

I'd invest in the extra lines of code...

[Falmarri]

Just in case anyone cares, which I'm sure no one does, I DID in fact end up using the python join and abspath and such commands.

[Falmarri]

I agree, but the filename is part of a command string, and instead of parsing out the command string to append to the filename (at the end), I'd just like to append it.

[Gilles 'SO- stop being evil']

@Falmarri: You can't just append a file name to a command string! A command string will be parsed by the shell, so special characters in file names need to be quoted. So you do need to construct the file name, then quote it properly to put it into the command string.

[Falmarri]

This is a really specific project that I'm pretty much just going to be using myself. I probably haven't been clear enough to justify not being robust about this. I'm getting this file path string from a class that gives me a correctly escaped file path and such. And I'm appending it to a command line argument

[Balakrishnan]

@Falmarri: So use normpath to clean up the command-line value that you don't control, and then use join to put them together.

[Falmarri]

This is actually what I ended up doing =\ I couldn't handle the special case where I was just given / very well.

[Michael Mrozek]

There can be if it's exactly two and at the beginning; A pathname that begins with two successive slashes may be interpreted in an implementation-defined manner. In practice I think this is right and they just get ignored

[BCoates]

Thanks Chris, I appreciate the clarification! (unfortunately the OpenID login isn't working for me or I would vote you up)

[Michael Mrozek]

@Rob You're unregistered, but still logged in (you're tracked by your cookie). You should be able to register now to connect an OpenID to your account, but you should be able to vote either way

[ephemient]

Cygwin (while not a real UNIX) does translate //remote/... to remote filesystem access, probably for consistency with Windows' \\remote\....

[BCoates]

Thanks Michael but "you have to be log in or register to vote". When you use just an email address and name you don't have full privileges. And since OpenID is timing out and I don't feel like creating another account, I'm out of luck. My fault for being lazy I guess, but I appreciate the help.

[Stephen P]

I believe (but can't google up a good reference right now) that Windows POSIX compat APIs will also treat //remote/... the same as the UNC path \\remote\... format.

[James]

I think I recall that Boost.Filesystem's portable pathnames handle // in a special way, in that they may test false for being absolute, conforming to the Unix/POSIX spec.

[hakre]

The equivalent to /. has been removed after a later discussion process as it was ambiguous. Anyway +1 as finding this kind of information well summarized is hard.

[Bluehorn]

Beware the following comment in the source of normpath: Normalize a path, e.g. A//B, A/./B and A/foo/../B all become A/B. It should be understood that this may change the meaning of the path if it contains symbolic links!

[Stéphane Chazelas]

See also: On what systems is //foo/bar different from /foo/bar?