Apache Commons File Upload - Stream ended unexpectedly

Question

Well, I have to say that so far, this one has me stumped. Our web application, which is running in Tomcat 6.0.18 is failing during file upload, but only when the client machine is a windows machine, only for some machines, and for all browsers, not just IE.

There is a stack trace in the logs, which seems to indicate that the client either closed the connection, or the stream was somehow corrupted. The root cause in the stack trace is given as follows:

Caused by: org.apache.commons.fileupload.MultipartStream$MalformedStreamException: Stream ended unexpectedly
    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.makeAvailable(MultipartStream.java:983)
    at org.apache.commons.fileupload.MultipartStream$ItemInputStream.read(MultipartStream.java:887)
    at java.io.InputStream.read(InputStream.java:85)
    at org.apache.commons.fileupload.util.Streams.copy(Streams.java:94)
    at org.apache.commons.fileupload.util.Streams.copy(Streams.java:64)
    at org.apache.commons.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:362)
    ... 70 more

The code which causes the trace looks fairly straight forward.

private Map<String, Object> getMap( ActionRequest request ) {

    HashMap<String, Object> parameters = new HashMap<String, Object>();
    if ( request == null ) {
        return parameters;
    }

    if ( request.getContentType() == null ) {
        return parameters;
    }

    try {
        if(PortletFileUpload.isMultipartContent(request)){
            DiskFileItemFactory factory = new DiskFileItemFactory();
            PortletFileUpload upload = new PortletFileUpload(factory);
            List<DiskFileItem> fileItems = upload.parseRequest(request);
            for( DiskFileItem fileItem : fileItems ) {
                String name = fileItem.getFieldName();
                //now set appropriate variable, populate hashtable
                if( fileItem.isFormField() ) {
                    String value = fileItem.getString( request.getCharacterEncoding() );
                    if( parameters.get( name ) == null ) {
                        String[] values = new String[1];
                        values[0] = value;
                        parameters.put( name, values );
                    } else {
                        Object prevobj = parameters.get( name );
                        if( prevobj instanceof String[] ) {
                            String[] prev = ( String[] ) prevobj;
                            String[] newStr = new String[prev.length + 1];
                            System.arraycopy(
                                    prev, 0, newStr, 0,
                                    prev.length
                            );
                            newStr[prev.length] = value;
                            parameters.put( name, newStr );
                        } else {
                            //now what? I think this breaks the standard.
                            throw new EatMyHatException(
                                    "file and input field with same name?"
                            );
                        }
                    }
                } else {
                    // Yes, we don't return FileParameter[] for multiple files of same name.  AFAIK, that's not allowed.
                    FileParameter fp = new FileParameter( fileItem );
                    parameters.put( name, fp );
                    files.add( fp );
                }
            }
        } else {
            // Not multipart
            return toObjectMap(request.getParameterMap());
        }
    } catch (FileUploadException e) {
        throw new RuntimeException(e);
    } catch (UnsupportedEncodingException e) {
        throw new RuntimeException(e);
    }
    return parameters;
}

The line which is giving us the grief is this one:

List<DiskFileItem> fileItems = upload.parseRequest(request);

Which for some reason is deciding that streams from some Windows machines are in some way corrupted.

I think I have found something that may be related on StackOverflow. It seems to suggest there is some bug in Tomcat 6 which was fixed in version 6.0.20, a slightly higher version than the one we're using. Unfortunately it doesn't mention what the issue itself was. I've had a look at the Tomcat changelog, but can't see any likely candidates for a bug which could cause this problem.

Anyway, on to my actual question, has anyone encountered a similar problem, and if so, what was the underlying issue and how did you resolve it?

Thank you in advance for any responses.

EDIT: This appears to be some kind of problem with load balancing and Tomcat. If you bypass the load balancer and access Tomcat directly via the server IP address, the problem goes away. The strange thing is that this appears in both our staging environment, in which we use Apache/AJP1.3, and live, where we use Zeus.

EDIT3: This turned out to be an issue with the clients firewall. It appears that they were.. er.. not being entirely truthful when they said that the knew definitely this was not a Firewall issue.

Solution

May be you need tcpdump/wireshark bad and right uploads, and then compare them?