Content-Length not sent when gzip compression enabled in Apache?

Question

I'm having trouble returning a city using reverse geocoding in Objective C on iOS. I'm able to log the city within the completionHandler, but I can't seem to figure out how to return it as a string if it's called from another function.

The city variable is an NSString created in the header file.

- (NSString *)findCityOfLocation:(CLLocation *)location
{

    geocoder = [[CLGeocoder alloc] init];
    [geocoder reverseGeocodeLocation:location completionHandler:^(NSArray *placemarks, NSError *error) {

        if ([placemarks count])
        {

            placemark = [placemarks objectAtIndex:0];

            city = placemark.locality;

        }
    }];

    return city;

}

Solution 1

Addition to Martin Fjordvalds answer:

Apache uses chunked encoding only if the compressed file size is larger than the DeflateBufferSize. Increasing this buffer size will therefore prevent the server using chunked encoding also for larger files, causing the Content-Length to be sent even for zipped data.

More Information is available here: http://httpd.apache.org/docs/2.2/mod/mod_deflate.html#deflatebuffersize

Solution 2

Sounds like Apache is doing chunked encoding, this means it can send the data as it's being gzipped rather than waiting for the full response to be gzipped. It's fairly standard practice, I'm not familiar enough with Apache to say if it can be disabled, though.

Solution 3

OK, I managed to solve this. As Martin F correctly points out, Apache is chunking the reply so the content size is not known. For many people this is desirable (page loads faster). This comes at a cost of not being able to report the download progress.

For those like me who really want to report the download progress, if you use Apache or PHP's automatic gzip support, there is little you can do. The solution is to do it manually. It's easier than it sounds:

If you're sending whole files, then this is a great example in PHP to force a single chunk (with the Content-Length): http://www.php.net/manual/en/function.ob-start.php#94741

If you're sending generated data, then use gzencode to encode your data, like in the above sample. A pre-requisite is that all your output data is stored in a variable (you can use ob_start to help this if you need to buffer, then get contents of buffer).

        // $replyBody is the entire contents of your reply

        header("Content-Type: application/json");  // or whatever yours is

        // checks if gzip is supported by client
        $pack = true;
        if(empty($_SERVER["HTTP_ACCEPT_ENCODING"]) || strpos($_SERVER["HTTP_ACCEPT_ENCODING"], 'gzip') === false)
        {
            $pack = false;
        }

        // if supported, gzips data
        if($pack) {
            header("Content-Encoding: gzip");
            $replyBody = gzencode($replyBody, 9, FORCE_GZIP);
        }

        // compressed or not, sets the Content-Length           
        header("Content-Length: " . mb_strlen($replyBody, 'latin1'));

        // outputs reply & exits
        echo $replyBody;
        exit;

And voila!

Another great benefit of doing it yourself is that you can set the compression level. This is great for my mobile application, as I can set to the highest compression level (so my users pay less for data!) – whereas the server probably only uses a medium compression level for a better CPU/size trade-off. Compression levels are something I believe you can only change if you can edit the httpd.conf (which on shared hosting, I can't).

So I've kept my DEFLATE .htaccess directive for everything but my application/json replies which I now encode in the above way.

Thanks again Martin F, you gave me the spark I needed to solve this :)

Comments

[William Denniss]

Thanks for the info, you pointed me in the right direction, and I solved it.

[William Denniss]

Incidentally, the savings with JSON data (with heavily repeated keys) are huge, 77% reduction in one case. That's a big deal at $1 per MB...

[William Denniss]

Accepted. For anyone reading this question though – please read my answer for a detailed solution. Basically, you can avoid chunking (and thus the zero content-length) by buffering and compressing the reply manually.

[redbmk]

It's a little confusing that the accepted answer isn't the answer to the original question, but rather something that helped you get it. Maybe you should accept the answer you posted below to make things a little more clear.

[William Denniss]

Nice one. This is probably the fastest way to solve this problem. If anyone needs a higher level of customisation (e.g. chunk some requests, not others), see my answer serverfault.com/a/183856/54957 for a manual solution.

[William Denniss]

@redbmk fair point, I just didn't want to seem ungrateful. Philippe actually has the perfect simple fix for this, so I've accepted his over mine.

[Gabriele Petronella]

You're welcome. Even though you cannot up vote the answer, you can accept it by selecting the tick under the score ;)

[maxmclau]

Sorry about that, I didn't know could only check off a single answer. Fixed