Use FallbackResource even if directory exists

Question

I've set up my virtual host on Apache 2.4.7 with a very basic configuration:

<VirtualHost *:80>
  ServerName foo.example.com
  DocumentRoot /var/www/html

  DirectoryIndex index.php
  FallbackResource /index.php
</VirtualHost>

Underneath the document root I have the following structure:

/index.php
/help/readme.txt

I get the following results when I make requests:

/bla     -> 200 OK
/help/   -> 404 Not Found
/help/a  -> 200 OK

It seems that the existence of the /help/ directory is causing Apache to return 404 because there's no index.php in there, but I expect all requests to invoke /index.php and thus yield a 200 OK response.

I don't remember this being a problem when using mod_rewrite, but I prefer using FallbackResource if possible. Is there a way to fix this?

Update

It works if I remove the DirectoryIndex directive, but that suffers from five second delay issues.

Update 3

I'm running the following test environment; the directory structure is as follows:

./htdocs
   index.html
   test/
      bla.txt
./conf
   httpd.conf
./logs

The contents of httpd.conf is:

ServerName apache-bug.local
Listen 8085

DirectoryIndex disabled
DirectorySlash Off

<VirtualHost *:8085>
DocumentRoot /home/user/apache-bug/htdocs

FallbackResource /index.html
</VirtualHost>

My config.nice contains:

"./configure" \
"--enable-debugger-mode" \
"--with-apr=/usr/local/apr/bin/apr-1-config" \
"--enable-dir=static" \
"--with-mpm=prefork" \
"--enable-unixd=static" \
"--enable-authn-core=static" \
"--enable-authz-core=static" \
"$@"

To run the server:

httpd -X -d /home/user/work/apache-bug/

Solution 1

I'm answering this myself as well, because I'm pretty sure that the issue is related to how mod_dir.c works internally and I think it's a bug.

If a resource can't be mapped to the local file system, the function fixup_dflt() will run, using the FallbackResource to determine which document should be loaded instead.

However, when a resource can be mapped to the local file system and it's a directory, it will attempt to resolve the document by running fixup_dir(); this function iterates over the list of DirectoryIndex values until it finds the first suitable document.

In my case, the configuration has an empty list of DirectoryIndex values, so fixup_dir() will fail and a 404 is returned.

The following patch works for me (PR):

static int dir_fixups(request_rec *r)
{
    if (r->finfo.filetype == APR_DIR) {
-        return fixup_dir(r);
+        if (fixup_dir(r) != OK) {
+           /* use fallback */
+           return fixup_dflt(r);
+        }
+
+        return OK;
    }
    else if ((r->finfo.filetype == APR_NOFILE) && (r->handler == NULL)) {
        /* No handler and nothing in the filesystem - use fallback */
        return fixup_dflt(r);
    }
    return DECLINED;
}

It basically tries fixup_dflt() after fixup_dir() failed.

Update 2015-04-21

A fix has been submitted to the project, scheduled for 2.5; it may be ported to 2.4 as well.

Update 2015-05-18

The fix has been reverted because:

[...] it [at least] causes FallBackResource to kick in before mod_autoindex might have kicked in.

I'm still trying to figure out how to avoid this kind of situation.

Solution 2

Your configuration should be correct.

The problem, strangely enough, seems to be mod_deflate.

After having reproduced your configuration successfully here (not getting a 404), I also got the 5 second delay. However, I noticed that when the UA omits gzip from its Accept-Headers, the page is displayed/received instantaneously. You can test this for yourself with wget.

Interestingly, further debugging with strace shows that apache sends the contents of your FallbackResource to the socket of your client without noticeable difference in delay for both cases. This is also evident on the wire, where a reply paket is sent from the server to the client after the HTTP Request without any notable delay¹.

It seems that when using mod_deflate in this case however, the UA doesn't know when the data sent by the server ends, and thus does not render anything before the TCP connection times out² and is forcibly closed by the server. This is compliant to HTTP/1.0, where a closed connection signifies end-of-content.

For HTTP/1.1, the server has other means available to signal the end-of-content – but none of which seem to happen here.

Wether the bug lurks in mod_dir or mod_deflate however, is beyond my available time right now. I got it to work flawlessy by disabling gzip compression; as a workaround until the issue is fixed for good, you can selectively disable gzip.

¹) This tells us that the problem does not stem from unflushed buffers on the server.
²) By default, the timeout is 5 seconds with apache - that's where your 5 seconds are from.

Comments

[zerkms]

And what is in the response body for /bla?

[Jenny D]

Just to make sure - which version of Apache are you on?

[Jack]

@JennyD I'm running on 2.4.7.

[Jack]

Thanks. The 5s delay issue is secondary to my main problem, I've updated my question with how to reproduce the issue locally.

[Ricky]

I would say the error is in fixup_dir() ignoring the FallbackResource.

[Jack]

@RickyBeam Yes, but technically fixup_dir() shouldn't know about fixup_dflt(), so imho it's better to fix it "higher up" :)