How to pass large quantity of data to web service

Question

I'm building a client-server (c#) application that uses a web services to synchronize the data.

Basically I am passing the XML of a DataSet back and forth. But depending on various parameters the dataset can actually be quite large. I'm compressing the XML using Gzip and want to pass that to the web server and the get the resulting compressed XML back.

What is the best way to pass potentially large chunks of data back and forth?

Clarification: I guess I'm asking what format is the best to pass the data. JSON, SOAP, ordinary POST (I'm not very familiar with Web Services so I'm sure there's more that I'm not thinking of).

Solution 1

Best probably depends on a lot of factors. If by best you mean most performant, here are some points to consider:

• XML is not the best way at all
• Binary serialization is far more efficient.
• You may not have this option, however, if you need to concern yourself with interoperability. In that case, you may want to consider using a flat file or delimited format.
• If neither of those are doable, then you might consider just sending what has changed instead of all of the data.
• If that isn't an option, then compressing the XML is an... ok solution.

I'd use XML as a last resort, since there's a lot of unnecessary metadata that is included in XML that will easily increase the size of your payload by more than 5X.

Solution 2

What you probably want to investigate is using MTOM. This allows you to send binary SOAP in a chunked manner. This is cool because chunking the data, coupled with an AJAX UI, will allow you to present a progress bar to the user that is uploading it.

Here is an example of using MTOM to send/receive data in a chunked manner. I hope this helps.