Azure Websites - 502 - Web server received an invalid response while acting as a gateway or proxy server

Question

My web app is running fine on my local machine but when published to a Windows Azure Website, I get the following error on one of the controller action methods:

502 - Web server received an invalid response while acting as a gateway or proxy server. There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.

I used Elmah to capture more information and I get: System.ArgumentNullException Value cannot be null. Parameter name: path2

So it's referencing when I use path.combine(path1, path2) in my method. I can't figure out what is going on, the input file is being read fine and the output files are being generated fine when I check the output directory.

Any other suggestions on what might be going on?

Here is the code for my action method:

        [HttpPost]
        public ActionResult ProcessProducts(UploadViewModel model)
        {
            DateTime startTime = DateTime.UtcNow;
            DateTime endTime;
            TimeSpan totalTime;            
            PulProcessor prodManager = new PulProcessor();
            string filePath = Path.Combine(Server.MapPath("~/files/incoming"), model.ProductsFileName);

            try
            {                
                using (TextReader prodFile = System.IO.File.OpenText(filePath))
                {
                    CsvReader csv = new CsvReader(prodFile);
                    // map is at end of this file
                    csv.Configuration.RegisterClassMap<PulMap>();
                    List<PulProduct> prodList = csv.GetRecords<PulProduct>().ToList();

                    foreach (var product in prodList)
                    {
                        prodManager.ProcessProduct(product);
                    }
                }
            }
            catch (Exception ex)
            {
                Elmah.ErrorSignal.FromCurrentContext().Raise(ex);
            }
            string currentDate = DateTime.UtcNow.ToString("yyyyMMdd");
            string productFileName = "PUL" + currentDate + ".txt";
            string exceptionsFileName = "PUL" + currentDate + "belowcost.txt";

            WriteFile(prodManager.Products, productFileName);
            WriteFile(prodManager.BelowCost, exceptionsFileName);

            endTime = DateTime.UtcNow;
            totalTime = endTime - startTime;

            ViewBag.StartTime = startTime.ToString();
            ViewBag.EndTime = endTime.ToString();
            ViewBag.TotalTime = totalTime.ToString();
            ViewBag.TotalOutput = prodManager.Products.Count.ToString();
            ViewBag.ProductCounter = prodManager.RecordsProcessed;
            ViewBag.FileName = productFileName;
            ViewBag.ExFileName = exceptionsFileName;
            ViewBag.Exceptions = prodManager.BelowCost.Count.ToString();

            return View();
        }

    private void WriteFile(List<PulFinalProduct> prodList, string fileName)
    {
        try
        {
            string filePath = Path.Combine(Server.MapPath("~/files/pul"), fileName);
            StreamWriter writer = new StreamWriter(filePath, false);
            StringBuilder fileHeader = new StringBuilder();

            fileHeader.Append("Inventory Number\t");
            fileHeader.Append("MPN\t");
            fileHeader.Append("Retail Price\t");
            fileHeader.Append("Seller Cost\t");
            fileHeader.Append("Buy It Now Price\t");
            fileHeader.Append("Starting Bid\t");
            fileHeader.Append("ChannelAdvisor Store Price\t");
            fileHeader.Append("Quantity\t");
            fileHeader.Append("Quantity Update Type\t");
            fileHeader.Append("UPC\t");
            fileHeader.Append("Weight\t");
            fileHeader.Append("Brand\t");
            fileHeader.Append("Manufacturer\t");

            using (writer)
            {
                writer.WriteLine(fileHeader.ToString());

                foreach (var product in prodList)
                {
                    StringBuilder productLine = new StringBuilder();
                    productLine.Append(product.InventoryNumber + "\t");
                    productLine.Append(product.MPN + "\t");
                    productLine.Append(product.RetailPrice.ToString() + "\t");
                    productLine.Append(product.SellerCost.ToString() + "\t");
                    productLine.Append(product.BINPrice.ToString() + "\t");
                    productLine.Append(product.StartingBid.ToString() + "\t");
                    productLine.Append(product.CAStorePrice + "\t");
                    productLine.Append(product.Quantity + "\t");
                    productLine.Append(product.QUType + "\t");
                    productLine.Append(product.UPC + "\t");
                    productLine.Append(product.Weight + "\t");
                    productLine.Append(product.Brand + "\t");
                    productLine.Append(product.Manufacturer + "\t");

                    writer.WriteLine(productLine.ToString());
                }
            }
        }
        catch (Exception ex)
        {
            Elmah.ErrorSignal.FromCurrentContext().Raise(ex);
        }
    }

Here is what the Azure Eventlog gives me:

Value cannot be null. Parameter name: path2 at System.IO.Path.Combine(String path1, String path2) at Jemco.Web.Controllers.PartsUnlimitedController.ProcessProducts(UploadViewModel model) at lambda_method(Closure , ControllerBase , Object[] ) at System.Web.Mvc.ActionMethodDispatcher.Execute(ControllerBase controller, Object[] parameters) at System.Web.Mvc.ReflectedActionDescriptor.Execute(ControllerContext controllerContext, IDictionary`2 parameters) at System.Web.Mvc.ControllerActionInvoker.InvokeActionMethod(ControllerContext controllerContext, ActionDescriptor actionDescriptor, IDictionary`2 parameters) at System.Web.Mvc.Async.AsyncControllerActionInvoker.<BeginInvokeSynchronousActionMethod>b__36(IAsyncResult asyncResult, ActionInvocation innerInvokeState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`2.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3c() at System.Web.Mvc.Async.AsyncControllerActionInvoker.AsyncInvocationWithFilters.<>c__DisplayClass45.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3e() at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass30.<BeginInvokeActionMethodWithFilters>b__2f(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<>c__DisplayClass28.<BeginInvokeAction>b__19() at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass1e.<BeginInvokeAction>b__1b(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeAction(IAsyncResult asyncResult) at System.Web.Mvc.Controller.<BeginExecuteCore>b__1d(IAsyncResult asyncResult, ExecuteCoreState innerState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult) at System.Web.Mvc.Controller.<BeginExecute>b__15(IAsyncResult asyncResult, Controller controller) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult) at System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult) at System.Web.Mvc.MvcHandler.<BeginProcessRequest>b__4(IAsyncResult asyncResult, ProcessRequestState innerState) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncVoid`1.CallEndDelegate(IAsyncResult asyncResult) at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResultBase`1.End() at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult) at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result) at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) 

OK, I was able to do remote debugging on this azure site. The code inside my controller action is running fine. After I get the original 502 error, the controller continues to process the input data and then write the output files. Everything works fine and the only problem I'm having is getting a 502 page while I am waiting for it to process and load the next view. Does a controller have a certain amount of time to return a view and if it doesn't is this why I'm getting the 502 error?

EDIT: I'm pretty sure what I'm experiencing is a timeout error because my controller method takes too long to run. I probably can't speed up the controller method so I'm not sure what I can do. I don't think I can or should change timeout settings on the server so maybe there something using ajax that can be done like sending a status update to the browser after every so often. I'm totally new to this so I'll have to do some research.

Solution

It turned out that the whole subroutine was taking way too long to function and was timing out hence the 502 error. I solved this by re-writing the class to use SignalR to run the task to create the file and then update the user as to the progress that was being made.