Why does Apache (and other browsers) throw 403 due to file system permissions?

Question

To forestall any confusion, I am not asking for help with a technical issue debugging my web server. I am asking about the history and logic behind Apache (and other web servers) throwing 403 when they don't have permissions on a file or its parent directory.

e.g., suppose I have a file like this in my DocumentRoot:

-rw-r----- 1 nobody staff 0 Apr 12 09:35 file.html

And Apache runs under the user www who is not part of the staff group. If I try to open the URL for this file, I get HTTP 403 Forbidden.

The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated.

I understand the technical reason why Apache cannot serve the file. But why use 403? Would not one of the 500 family of errors be more accurate (or 404)?

My reasoning here is that the server is failing to serve the file because the file system permissions are either incorrect (misconfigured) or the admin intentionally wants the file in the document tree but not accessible to the web (think about it.. the error message often says "You don't have permission to access /file.html on this server.".. well who does? Is Apache configured to switch OS users based on who's logged into the web interface?).

If the file is outside of the document tree (e.g., /etc/passwd), then I would expect the server to give 404.

edit: writing/grammar

Solution

For historic reasons simply look at the older HTTP 1.0 specification from RFC 1945 (emphasis mine):

#403 Forbidden

The server understood the request, but is refusing to fulfill it.
Authorization will not help and the request should not be repeated.
If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it should describe the reason for the refusal in the entity body.
This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.

So a then 403 error was simply both convention and provides an error message that does not disclose unnecessary information.

In the much more recent RFC 7231 a 403 error becomes a much more generic "unauthorized", i.e. the user does not have the necessary permissions for the resource:

... If authentication credentials were provided in the request, the server considers them insufficient to grant access. The client SHOULD NOT automatically repeat the request with the same credentials. The client MAY repeat the request with new or different credentials. However, a request might be forbidden for reasons unrelated to the credentials.

Comments

[FooBee]

Server Fault is not about "Let's discuss this random topic I thought about". We deal with answerable questions about actual technical problems you face.

[Exit42]

Okay thanks @Sven, I suspected as much but didn't know where to ask. Where do I put it?

[FooBee]

Nowhere on Stack Exchange, because all sites are Q&A sites, not discussion forums. Other than that, I can't say.

[Exit42]

Thanks. I appreciate the advice. However, I find you describing my question with the phrase "Let's discuss this random topic I thought about" a bit negative.. perhaps condescending or belittling. I didn't select this topic randomly or arbitrarily or frivolously. I asked it because I seek to understand the technology that I work with. Chalk it up to me being over-sensitive, if you like, but I don't think your comment had the appropriate level of constructiveness.

[FooBee]

From my point of view, it is just another random question that has no place here. I really think it's a good question in itself, but it won't solve an actual problem for you or change the way you operate your environment, which is one of the most fundamental requirements for questions here. Ignoring this kind of requirement, even when you don't know where else to ask, is widely considered as disrespectful against any online community whose time you wish to occupy for free, so please just accept that I won't sugarcoat my being a little bit annoyed with a metaphorical fake smile.

[Exit42]

Makes sense, I get it.