RestSharp print raw request and response headers

Question

I'm using RestSharp to make calls to a webservice. All is well but I was wondering if it would be possible to print the raw request headers and body that is sent out and the raw response headers and the response body that comes back.

This is my code where I create a request and get a response back

public static TResponse ExecutePostCall<TResponse, TRequest>(String url, TRequest requestData, string token= "") where TResponse : new()
{
    RestRequest request = new RestRequest(url, Method.POST);
    if (!string.IsNullOrWhiteSpace(token))
    {
        request.AddHeader("TOKEN", token);
    }
    request.RequestFormat = DataFormat.Json;
    request.AddBody(requestData);
    // print raw request here
    var response = _restClient.Execute<TResponse>(request);
    // print raw response here
    return response.Data;
}

so, Would it be possible to print the raw request and response?

Solution 1

RestSharp doesn't provide a mechanism to achieve exactly what you want and activating the .Net tracing is a bit overkilling IMO.

For logging (debugging) purposes (something that I can leave turned on for a while in PROD for example) I have found this approach to be very useful (although it has some details on how to call it, read below the code):

private void LogRequest(IRestRequest request, IRestResponse response, long durationMs)
{
        var requestToLog = new
        {
            resource = request.Resource,
            // Parameters are custom anonymous objects in order to have the parameter type as a nice string
            // otherwise it will just show the enum value
            parameters = request.Parameters.Select(parameter => new
            {
                name = parameter.Name,
                value = parameter.Value,
                type = parameter.Type.ToString()
            }),
            // ToString() here to have the method as a nice string otherwise it will just show the enum value
            method = request.Method.ToString(),
            // This will generate the actual Uri used in the request
            uri = _restClient.BuildUri(request),
        };
        var responseToLog = new
        {
            statusCode = response.StatusCode,
            content = response.Content,
            headers = response.Headers,
            // The Uri that actually responded (could be different from the requestUri if a redirection occurred)
            responseUri = response.ResponseUri,
            errorMessage = response.ErrorMessage,
        };
        Trace.Write(string.Format("Request completed in {0} ms, Request: {1}, Response: {2}",
                durationMs, 
                JsonConvert.SerializeObject(requestToLog),
                JsonConvert.SerializeObject(responseToLog)));
}

Things to note:

• Headers, Url segments, QueryString parameters, body, etc. all are considered parameters for RestSharp, all that appear in the parameters collection of the request, with their corresponding type.
• The log method must be called AFTER the request took place. This is needed because of the way RestSharp works, the Execute method will add headers, run the authenticators (if some configured), etc. and all these will modify the request. So in order to have the all the real parameters sent logged, the Execute method should have been called before logging the request.
• RestSharp itself will never throw (instead errors get saved in the response.ErrorException property), but I think deserialization could throw (not sure) and besides I needed to log the raw response, so I chose to implement my own deserialization.
• Have in mind that RestSharp uses its own formatting when converting parameters values to generate the Uri, so serializing the parameters to log them may not show the exact same things that were put in the Uri. That's why the IRestClient.BuildUri method it's pretty cool to get the actually called Uri (including the base url, the replaced url segments, the added queryString parameters, etc).
• EDIT: Also have in mind that it could happen that the serializer RestSharp is using for the body is not the same this code is using, so I guess code could be adjusted to use request.JsonSerializer.Serialize() for rendering the body parameter (I haven't tried this).
• Some custom code was needed to achieve nice descriptions in the logs for the enums values.
• StopWatch usage could be moved around to include deserialization in the measuring.

Here it is a basic complete base class example with logging (using NLog):

using System;
using System.Diagnostics;
using System.Linq;
using NLog;
using Newtonsoft.Json;
using RestSharp;
namespace Apis
{
    public abstract class RestApiBase
    {
        protected readonly IRestClient _restClient;
        protected readonly ILogger _logger;
        protected RestApiBase(IRestClient restClient, ILogger logger)
        {
            _restClient = restClient;
            _logger = logger;
        }
        protected virtual IRestResponse Execute(IRestRequest request)
        {
            IRestResponse response = null;
            var stopWatch = new Stopwatch();
            try
            {
                stopWatch.Start();
                response = _restClient.Execute(request);
                stopWatch.Stop();
                // CUSTOM CODE: Do more stuff here if you need to...
                return response;
            }
            catch (Exception e)
            {
                // Handle exceptions in your CUSTOM CODE (restSharp will never throw itself)
            }
            finally
            {
                LogRequest(request, response, stopWatch.ElapsedMilliseconds);
            }
            return null;
        }
        protected virtual T Execute<T>(IRestRequest request) where T : new()
        {
            IRestResponse response = null;
            var stopWatch = new Stopwatch();
            try
            {
                stopWatch.Start();
                response = _restClient.Execute(request);
                stopWatch.Stop();
                // CUSTOM CODE: Do more stuff here if you need to...
                // We can't use RestSharp deserialization because it could throw, and we need a clean response
                // We need to implement our own deserialization.
                var returnType = JsonConvert.DeserializeObject<T>(response.Content);
                return returnType;
            }
            catch (Exception e)
            {
                // Handle exceptions in your CUSTOM CODE (restSharp will never throw itself)
                // Handle exceptions in deserialization
            }
            finally
            {
                LogRequest(request, response, stopWatch.ElapsedMilliseconds);
            }
            return default(T);
        }
        private void LogRequest(IRestRequest request, IRestResponse response, long durationMs)
        {
            _logger.Trace(() =>
            {
                var requestToLog = new
                {
                    resource = request.Resource,
                    // Parameters are custom anonymous objects in order to have the parameter type as a nice string
                    // otherwise it will just show the enum value
                    parameters = request.Parameters.Select(parameter => new
                    {
                        name = parameter.Name,
                        value = parameter.Value,
                        type = parameter.Type.ToString()
                    }),
                    // ToString() here to have the method as a nice string otherwise it will just show the enum value
                    method = request.Method.ToString(),
                    // This will generate the actual Uri used in the request
                    uri = _restClient.BuildUri(request),
                };
                var responseToLog = new
                {
                    statusCode = response.StatusCode,
                    content = response.Content,
                    headers = response.Headers,
                    // The Uri that actually responded (could be different from the requestUri if a redirection occurred)
                    responseUri = response.ResponseUri,
                    errorMessage = response.ErrorMessage,
                };
                return string.Format("Request completed in {0} ms, Request: {1}, Response: {2}",
                    durationMs, JsonConvert.SerializeObject(requestToLog),
                    JsonConvert.SerializeObject(responseToLog));
            });
        }
    }
}

This class will log something like this (pretty formatted for pasting here):

Request completed in 372 ms, Request : {
    "resource" : "/Event/Create/{hostId}/{startTime}",
    "parameters" : [{
            "name" : "hostId",
            "value" : "116644",
            "type" : "UrlSegment"
        }, {
            "name" : "startTime",
            "value" : "2016-05-18T19:48:58.9744911Z",
            "type" : "UrlSegment"
        }, {
            "name" : "application/json",
            "value" : "{\"durationMinutes\":720,\"seats\":100,\"title\":\"Hello StackOverflow!\"}",
            "type" : "RequestBody"
        }, {
            "name" : "api_key",
            "value" : "123456",
            "type" : "QueryString"
        }, {
            "name" : "Accept",
            "value" : "application/json, application/xml, text/json, text/x-json, text/javascript, text/xml",
            "type" : "HttpHeader"
        }
    ],
    "method" : "POST",
    "uri" : "http://127.0.0.1:8000/Event/Create/116644/2016-05-18T19%3A48%3A58.9744911Z?api_key=123456"
}, Response : {
    "statusCode" : 200,
    "content" : "{\"eventId\":2000045,\"hostId\":116644,\"scheduledLength\":720,\"seatsReserved\":100,\"startTime\":\"2016-05-18T19:48:58.973Z\"",
    "headers" : [{
            "Name" : "Access-Control-Allow-Origin",
            "Value" : "*",
            "Type" : 3
        }, {
            "Name" : "Access-Control-Allow-Methods",
            "Value" : "POST, GET, OPTIONS, PUT, DELETE, HEAD",
            "Type" : 3
        }, {
            "Name" : "Access-Control-Allow-Headers",
            "Value" : "X-PINGOTHER, Origin, X-Requested-With, Content-Type, Accept",
            "Type" : 3
        }, {
            "Name" : "Access-Control-Max-Age",
            "Value" : "1728000",
            "Type" : 3
        }, {
            "Name" : "Content-Length",
            "Value" : "1001",
            "Type" : 3
        }, {
            "Name" : "Content-Type",
            "Value" : "application/json",
            "Type" : 3
        }, {
            "Name" : "Date",
            "Value" : "Wed, 18 May 2016 17:44:16 GMT",
            "Type" : 3
        }
    ],
    "responseUri" : "http://127.0.0.1:8000/Event/Create/116644/2016-05-18T19%3A48%3A58.9744911Z?api_key=123456",
    "errorMessage" : null
}

Hope you find this useful!

Solution 2

.net provides its own yet powerful logging feature. This can be turned on via config file.

I found this tip here. John Sheehan pointed to How to: Configure Network Tracing article. (a note: I edited the config provided, turned off unnecessary (for me) low level logging).

  <system.diagnostics>
    <sources>
      <source name="System.Net" tracemode="protocolonly" maxdatasize="1024">
        <listeners>
          <add name="System.Net"/>
        </listeners>
      </source>
      <source name="System.Net.Cache">
        <listeners>
          <add name="System.Net"/>
        </listeners>
      </source>
      <source name="System.Net.Http">
        <listeners>
          <add name="System.Net"/>
        </listeners>
      </source>
    </sources>
    <switches>
      <add name="System.Net" value="Verbose"/>
      <add name="System.Net.Cache" value="Verbose"/>
      <add name="System.Net.Http" value="Verbose"/>
      <add name="System.Net.Sockets" value="Verbose"/>
      <add name="System.Net.WebSockets" value="Verbose"/>
    </switches>
    <sharedListeners>
      <add name="System.Net"
        type="System.Diagnostics.TextWriterTraceListener"
        initializeData="network.log"
      />
    </sharedListeners>
    <trace autoflush="true"/>
  </system.diagnostics>

Solution 3

I just found the code below in the RestSharp examples. It allows you to print your raw response.

client.ExecuteAsync(request, response =>
                   {
                       Console.WriteLine(response.Content);
                   });

Solution 4

You have to loop through the request.Parameters list and format it to a string in whatever format you like.

var sb = new StringBuilder();
foreach(var param in request.Parameters)
{
    sb.AppendFormat("{0}: {1}\r\n", param.Name, param.Value);
}
return sb.ToString();

If you want the output to show request headers and then the body similar to Fiddler, you just need to order the collection by Request headers and then by Request body. The Parameter object in the collection has a Type parameter enum.

Solution 5

You can use Fiddler for capturing HTTP requests.

View more solutions

Comments

[wal]

do you want to do this every time or just to debug something? if just a one-off then use fiddler to get the raw requests going back and forth

[NilsH]

Not a complete answer, but you can write your own serializer/deserializer and log the genereated/consumed JSON there. But you might be better off with a "sniffing" proxy as suggested above.

[Professor Chaos]

@wal I have been using fiddler. I want to do this everytime in my .net app.

[wal]

do u need the body or just the headers? 'everything' seems a bit overkill but i dont know what it is you want exactly.

[Professor Chaos]

@wal I definitely want the HTTP response code and the response body.. for request, I need the method and the url and the request body.

[Alex.F]

Trace.WriteLine(request.JsonSerializer.Serialize(request)) will not give you the raw request, all it will do is serialize the object, which is a completely different thing.

[Juri]

Note: please removed white space in listener name - otherwise diagnostics throws "shared listener not found exception".

[The Muffin Man]

Sometimes it's nice to be able to log these types of things when trying to troubleshoot an issue that a client is having without installing Fiddler.

[Joshua Enfield]

Output will show in the "Debug" feed of the output window with the above configuration. Be sure to open it prior to starting debugging.

[Konstantin Chernov]

very useful and compact

[David Keaveny]

I modified your code to be a decorator to the IRestClient interface instead of a base class, and use log4net instead of NLog. log4net needs a bit more help at writing out arrays of anonymous types, but otherwise it works great.

[LucasMetal]

@DavidKeaveny awesome! I'm not familiarized with adding decorators to the IRestClient interface, what class finally uses it? Do you have any link to check the implementation? Also check that I added an extra bullet point ;) Thanks for commenting.

[blogbydev]

How do we use fiddler here, do we need to change something in the config?

[Neshta]

@singsuyash Sorry, but I don't remember why I answered so year ago. I've read the question again and now I think that it is not a good answer. But I see that I voted this asnwer stackoverflow.com/a/21989958/1277458

[Kris Oye]

This does not work so well if you're using the Mono runtime, unfortunately.

[Tilendor]

Worjked great for me!

[Misiu]

Thank You. Works great! @LucasG.Devescovi could You share Your decorator code?

[LucasMetal]

I'm glad it worked for you! Regarding the decorator code, it was @DavidKeaveny the one who did it, I don't have it. Thanks ;)

[Andre]

I need to get the raw request value, exactly as sent, so I would like to know how implement the authenticator will help me, could you help? thanks

[Andre]

Does the Execute method will never throw? So can I safely log (request/response) after it, without any try catch surround? Thanks

[giganoide]

@Andre I added how you can implement the authenticator. Usually log the header and the body it's enough, otherwise it's still possible to access all the parameters of the request.

[LucasMetal]

Hi @Andre, according to the recommended usage in the RestSharp wiki, the Execute method will never throw, take a look at the text before the first example here: github.com/restsharp/RestSharp/wiki/Recommended-Usage . Anyway, the custom code that you put around it could throw, so depending on your use case you maybe want to try/catch it, take a look at my example class. Hope this helps ;)

[live-love]

Fiddler works, but sometimes you have to tweak the settings (ex. decrypt traffic)

[yallie]

There is a Nuget package for automatic logging of RestSharp requests and responses to Serilog: nuget.org/packages/RestSharp.Serilog.Auto

[tno2007]

RestSharp.Serilog.Auto is extremely heavy on the dependencies. I would not recommend it to anyone trying keep their project sizes down.

[user1641519]

This settings saved my day.