Httplistener and file upload

18,477

Solution 1

I think you are making things harder on yourself than necessary by doing this with an HttpListener rather than using the built in facilities of ASP.Net. But if you must do it this way here is some sample code. Note: 1) I'm assuming you're using enctype="multipart/form-data" on your <form>. 2) This code is designed to be used with a form containing only your <input type="file" /> if you want to post other fields or multiple files you'll have to change the code. 3) This is meant to be a proof of concept/example, it may have bugs, and is not particularly flexible.

static void Main(string[] args)
{
    HttpListener listener = new HttpListener();
    listener.Prefixes.Add("http://localhost:8080/ListenerTest/");
    listener.Start();

    HttpListenerContext context = listener.GetContext();

    SaveFile(context.Request.ContentEncoding, GetBoundary(context.Request.ContentType), context.Request.InputStream);

    context.Response.StatusCode = 200;
    context.Response.ContentType = "text/html";
    using (StreamWriter writer = new StreamWriter(context.Response.OutputStream, Encoding.UTF8))
        writer.WriteLine("File Uploaded");

    context.Response.Close();

    listener.Stop();

}

private static String GetBoundary(String ctype)
{
    return "--" + ctype.Split(';')[1].Split('=')[1];
}

private static void SaveFile(Encoding enc, String boundary, Stream input)
{
    Byte[] boundaryBytes = enc.GetBytes(boundary);
    Int32 boundaryLen = boundaryBytes.Length;

    using (FileStream output = new FileStream("data", FileMode.Create, FileAccess.Write))
    {
        Byte[] buffer = new Byte[1024];
        Int32 len = input.Read(buffer, 0, 1024);
        Int32 startPos = -1;

        // Find start boundary
        while (true)
        {
            if (len == 0)
            {
                throw new Exception("Start Boundaray Not Found");
            }

            startPos = IndexOf(buffer, len, boundaryBytes);
            if (startPos >= 0)
            {
                break;
            }
            else
            {
                Array.Copy(buffer, len - boundaryLen, buffer, 0, boundaryLen);
                len = input.Read(buffer, boundaryLen, 1024 - boundaryLen);
            }
        }

        // Skip four lines (Boundary, Content-Disposition, Content-Type, and a blank)
        for (Int32 i = 0; i < 4; i++)
        {
            while (true)
            {
                if (len == 0)
                {
                    throw new Exception("Preamble not Found.");
                }

                startPos = Array.IndexOf(buffer, enc.GetBytes("\n")[0], startPos);
                if (startPos >= 0)
                {
                    startPos++;
                    break;
                }
                else
                {
                    len = input.Read(buffer, 0, 1024);
                }
            }
        }

        Array.Copy(buffer, startPos, buffer, 0, len - startPos);
        len = len - startPos;

        while (true)
        {
            Int32 endPos = IndexOf(buffer, len, boundaryBytes);
            if (endPos >= 0)
            {
                if (endPos > 0) output.Write(buffer, 0, endPos-2);
                break;
            }
            else if (len <= boundaryLen)
            {
                throw new Exception("End Boundaray Not Found");
            }
            else
            {
                output.Write(buffer, 0, len - boundaryLen);
                Array.Copy(buffer, len - boundaryLen, buffer, 0, boundaryLen);
                len = input.Read(buffer, boundaryLen, 1024 - boundaryLen) + boundaryLen;
            }
        }
    }
}

private static Int32 IndexOf(Byte[] buffer, Int32 len, Byte[] boundaryBytes)
{
    for (Int32 i = 0; i <= len - boundaryBytes.Length; i++)
    {
        Boolean match = true;
        for (Int32 j = 0; j < boundaryBytes.Length && match; j++)
        {
            match = buffer[i + j] == boundaryBytes[j];
        }

        if (match)
        {
            return i;
        }
    }

    return -1;
}

To help you better understand what the code above is doing, here is what the body of the HTTP POST looks like:

Content-Type: multipart/form-data; boundary=----WebKitFormBoundary9lcB0OZVXSqZLbmv

------WebKitFormBoundary9lcB0OZVXSqZLbmv
Content-Disposition: form-data; name="my_file"; filename="Test.txt"
Content-Type: text/plain

Test
------WebKitFormBoundary9lcB0OZVXSqZLbmv--

I've left out the irrelevant headers. As you can see, you need to parse the body by scanning through to find the beginning and ending boundary sequences, and drop the sub headers that come before the content of your file. Unfortunately you cannot use StreamReader because of the potential for binary data. Also unfortunate is the fact that there is no per file Content-Length (the Content-Length header for the request specifies the total length of the body including boundaries, sub-headers, and spacing.

Solution 2

The problem is you are reading the file as text.

You need to read the file as a bytearray instead and using the BinaryReader is better and easier to use than StreamReader:

Byte[] bytes;
using (System.IO.BinaryReader r = new System.IO.BinaryReader(request.InputStream))
{
    // Read the data from the stream into the byte array
    bytes = r.ReadBytes(Convert.ToInt32(request.InputStream.Length));
}
MemoryStream mstream = new MemoryStream(bytes);

Solution 3

May have bugs, test thoroughly. This one gets all post, get, and files.

using System.Collections.Generic;
using System.Collections.Specialized;
using System.IO;
using System.Net;
using System.Text;
using System.Web;

namespace DUSTLauncher
{
    class HttpNameValueCollection
    {
        public class File
        {
            private string _fileName;
            public string FileName { get { return _fileName ?? (_fileName = ""); } set { _fileName = value; } }

            private string _fileData;
            public string FileData { get { return _fileData ?? (_fileName = ""); } set { _fileData = value; } }

            private string _contentType;
            public string ContentType { get { return _contentType ?? (_contentType = ""); } set { _contentType = value; } }
        }

        private NameValueCollection _get;
        private Dictionary<string, File> _files;
        private readonly HttpListenerContext _ctx;

        public NameValueCollection Get { get { return _get ?? (_get = new NameValueCollection()); } set { _get = value; } }
        public NameValueCollection Post { get { return _ctx.Request.QueryString; } }
        public Dictionary<string, File> Files { get { return _files ?? (_files = new Dictionary<string, File>()); } set { _files = value; } }

        private void PopulatePostMultiPart(string post_string)
        {
            var boundary_index = _ctx.Request.ContentType.IndexOf("boundary=") + 9;
            var boundary = _ctx.Request.ContentType.Substring(boundary_index, _ctx.Request.ContentType.Length - boundary_index);

            var upper_bound = post_string.Length - 4;

            if (post_string.Substring(2, boundary.Length) != boundary)
                throw (new InvalidDataException());

            var raw_post_strings = new List<string>();
            var current_string = new StringBuilder();

            for (var x = 4 + boundary.Length; x < upper_bound; ++x)
            {
                if (post_string.Substring(x, boundary.Length) == boundary)
                {
                    x += boundary.Length + 1;
                    raw_post_strings.Add(current_string.ToString().Remove(current_string.Length - 3, 3));
                    current_string.Clear();
                    continue;
                }

                current_string.Append(post_string[x]);

                var post_variable_string = current_string.ToString();

                var end_of_header = post_variable_string.IndexOf("\r\n\r\n");

                if (end_of_header == -1) throw (new InvalidDataException());

                var filename_index = post_variable_string.IndexOf("filename=\"", 0, end_of_header);
                var filename_starts = filename_index + 10;
                var content_type_starts = post_variable_string.IndexOf("Content-Type: ", 0, end_of_header) + 14;
                var name_starts = post_variable_string.IndexOf("name=\"") + 6;
                var data_starts = end_of_header + 4;

                if (filename_index == -1) continue;

                var filename = post_variable_string.Substring(filename_starts, post_variable_string.IndexOf("\"", filename_starts) - filename_starts);
                var content_type = post_variable_string.Substring(content_type_starts, post_variable_string.IndexOf("\r\n", content_type_starts) - content_type_starts);
                var file_data = post_variable_string.Substring(data_starts, post_variable_string.Length - data_starts);
                var name = post_variable_string.Substring(name_starts, post_variable_string.IndexOf("\"", name_starts) - name_starts);
                Files.Add(name, new File() { FileName = filename, ContentType = content_type, FileData = file_data });
                continue;

            }
        }

        private void PopulatePost()
        {
            if (_ctx.Request.HttpMethod != "POST" || _ctx.Request.ContentType == null) return;

            var post_string = new StreamReader(_ctx.Request.InputStream, _ctx.Request.ContentEncoding).ReadToEnd();

            if (_ctx.Request.ContentType.StartsWith("multipart/form-data"))
                PopulatePostMultiPart(post_string);
            else
                Get = HttpUtility.ParseQueryString(post_string);

        }

        public HttpNameValueCollection(ref HttpListenerContext ctx)
        {
            _ctx = ctx;
            PopulatePost();
        }


    }
}
Share:
18,477

Related videos on Youtube

cecemel
Author by

cecemel

Updated on June 04, 2022

Comments

  • cecemel
    cecemel over 1 year

    I am trying to retrieve an uploaded file from my webserver. As the client sends its files through a webform (random files), I need to parse the request to get the file out and to process it further on. Basically, the code goes as:

    HttpListenerContext context = listener.GetContext();
    HttpListenerRequest request = context.Request;
    StreamReader r = new StreamReader(request.InputStream, System.Text.Encoding.Default);
    // this is the retrieved file from streamreader
    string file = null;
    
    while ((line = r.ReadLine()) != null){
         // i read the stream till i retrieve the filename
         // get the file data out and break the loop 
    }
    // A byststream is created by converting the string,
    Byte[] bytes = request.ContentEncoding.GetBytes(file);
    MemoryStream mstream = new MemoryStream(bytes);
    
    // do the rest
    

    As a result, i am able to retrieve textfiles, but for all other files, they are corrupted. Could someone tell me how to parse these HttplistnerRequests properly (or providing a lightweighted alternative)?

  • cecemel
    cecemel almost 12 years
    Hi, thanks for the reply! It seems however that the request.InputStream.Length provides a not supported exception. Furthermore, bytearray complicates the job as there is some parsing that needs to take place to extract the file from other body content. Streamrider seemed from that perspective better fit for purpose.
  • competent_tech
    competent_tech almost 12 years
    @user1092608: Oh, that's unfortunate. The problem is that you can't really use the text read capabilities of streamreader to extract a binary file. Are you using a FileUpload control or some other method to embed the file in the stream?
  • cecemel
    cecemel almost 12 years
    That is a pertinent remark. Right now the client basically uses the default html <input type="file" name="datafile" ..> form. If possible, i would like to avoid the fileUpload control, to keep things as simple as possible -wathever this still means-. In this project, another instance (machine) should be able to throw random files at my server, along with some metadata, to do stuff with it and post the results back.
  • competent_tech
    competent_tech almost 12 years
    @user1092608: In this case, I would separate the receipt of the file from the processing into separate methods. I think the only way you are going to successfully receive a binary file from the user is to use the actual methods from the file input control. Once you have received the file, you can pass it to your common file-handling mechanism.
  • cecemel
    cecemel almost 12 years
    Hello, thank you for this elaborated response! It is indeed working and helps me a lot! (even if this is not the most efficient way)
  • fyasar
    fyasar almost 11 years
    Thats what I'm looking for. Thank you
  • mark vanzuela
    mark vanzuela almost 11 years
    solved my problem. But has to modify a bit if the stream has no boundary. Thanks a lot.
  • Mathias Conradt
    Mathias Conradt over 7 years
    @PaulWheeler Using your code above, the target files I transfer always have 2 bytes more than the original. This does not matter for jpg files, they still display fine, but zip files say that the CRC is not correct and when unzipping, some file is always corrupt. With all files, regardless of sizes, it's always 2 additional bytes. Any idea where they are coming from? (On Win8.1).
  • Mathias Conradt
    Mathias Conradt over 7 years
    Regarding my previous comment, note that I edited your answer/code, as per my finding in stackoverflow.com/questions/37744879/…
  • quest4truth
    quest4truth about 2 years
    This works great for situations where access to ASP.NET is not available. Thanks for showing us how to do it the hard way.