Putting detailed REST error message in HTTP Warning header, good/bad idea?

Question

We are developing a standard REST service using HTTP status codes as its response code if something went wrong. (e.g. invalid user input would return "400 Bad Request" to the client)

However, we felt that a more detailed error message would be useful for the client. (e.g. the invalid input error is due to X being a unrecognized parameter name)

We would like to be as faithful to the HTTP specs as possible, so after studying the specification in the RFC2616, we are thinking of putting the detailed error message in the HTTP Headers, specifically on the HTTP header warning field. It said on the RFC that:

The Warning general-header field is used to carry additional information about the status or transformation of a message which might not be reflected in the message. This information is typically used to warn about a possible lack of semantic transparency from caching operations or transformations applied to the entity body of the message.

There seems to be no restriction on using this header for other warnings (such as REST error message), even those that are unrelated with the cache warnings as per the original intention of this header. We like the semantic, and we planned to use the 299 warning code, which seems to fit the bill quite nicely:

299 Miscellaneous persistent warning The warning text MAY include arbitrary information to be presented to a human user, or logged. A system receiving this warning MUST NOT take any automated action.

So, given the invalid input error case presented on the top of this question, we're thinking of putting our REST error message like the following example:

HTTP/1.1 400 Bad Request
Warning: 299 ServiceName "Invalid input error: X is unrecognized parameter name."

Is this a good idea/practice? We also found that some services detailed this message in X-Warning header, but this seems to be not standard. We are wondering what would the hive wisdom of stackoverflow REST crowd think about this. Is there also any better/standardized practice for passing out detailed error messaging in REST responses?

Solution 1

Why not just change the reason phrase? That's what it is there for. The "Bad Request" text is just the default. If you want to include more information then use the response body. The HTTP spec says you SHOULD include a response body with details of an error.

---

UPDATE

Based on a more recent reading of RFC 7231 and related material, it appears the only valid reason for changing the reason phrase is to localize the text, not to provide a more specific meaning. Sorry about that.

Solution 2

Wherever you put your feedback, whether in the message body (content) or in a Warning header, be careful to avoid giving any information that might be helpful to an attacker doing penetration testing on your system.

Sometimes less info is better.

Solution 3

I'm in favor of using headers for warning only when the request succeeds in general.

For example a service that get's a user's details, but some of the details come from a third party which often goes down. In our case it was appropriate to leave that section of the user data blank, but show a warning to the user that some data is missing.

So the request returned a 200 Success with a payload that contained everything we could retrieve, but then had warning headers that described the error for the rest.

Solution 4

If this proposal is accepted, it presents an alternative for sending detail error messages. [https://datatracker.ietf.org/doc/html/draft-nottingham-http-browser-hints]

Despite it being an I-D, it's fairly stable lately, and I see no problem with building your own implementation. (I have done.)

Solution 5

I'm in favour of the general approach. We should assume that the client developer is in a different team from the service developer, maybe in a different timezone etc. Possibly even a different company. It's no good just returning a "no that's a bad request" response, how can the client fix the problem.

So philosophically: tell the client about the things they have a responsibility to fix. Errors that are purely the scope of the server (eg. database connection error, or some logical problem) it's fair just to return a 500 error. And here I would not send any detail back, we don't want to expose details of our internal implementation to the client.

Until now I've been returning data in the body of the response, using JAX/RS:

return Response.status(400).entity("some message here").build();

I'm thinking that your use of headers may actually be a cleaner appraoch.

View more solutions

Comments

[Barett]

Note: Warnings can accumulate between different requests in cache/proxy environments. Probably not an impact on a dynamic REST service, but something to keep in mind.

[dmportella]

what did you guys do in the end?

[Jim Grisham]

See also draft-cedik-http-warning and the issues discussion at http-warning at GitHub

[Jim Grisham]

There appears to be a standards-track RFC purpose-made for this sort of thing: RFC 7807 Problem Details for HTTP APIs

[fumanchu]

Do both. Changing the Reason-Phrase is good for machine clients, so e.g. you can write automated tests that check for specific conditions without having to parse HTML etc. And yes, a response body with details is a SHOULD, which means "do it if you want to call yourself fully interoperable".

[Ibrahim Arief]

Wow, brilliant approach, I honestly didn't know that we could customize the reason phrase. The legacy system we had right now didn't put any detailed error message in the response body, and we're constrained by some external legacy clients that might not expect a change in the body, so we tried to kept the external change as minimal as possible by sneaking the message in the header. Based on this suggestion, we end up implementing all three approaches, and let the output be customizable, many thanks!

[Jeffery Thomas]

Some clients (at least NSHTTPURLResponse) do not provide access to the reason phrase. Relying on it solely will cause problems for those clients. The warning header field should be universally accessible.

[Darrel Miller]

@JefferyThomas. There are some clients that don't let you access the HTTP headers at all, should we stop using those too?

[Jeffery Thomas]

If clients that have no access to the HTTP header are likely to use your system, then relying on it solely will cause problems for those clients. In that case the message body would need to be structured in such a way as to provide response status and detailed error messages. This doesn't mean you shouldn't use the HTTP header, you just should not rely on it solely. Just out of curiosity, which clients have you run into that do not have access to the HTTP header?

[Darrel Miller]

@JefferyThomas Yes, I phrased my question badly. I was just curious how far you would go in adding redundant information into your responses to support client software that can't be bothered to implement a spec that is 14 years old and supported by almost every network connected device on the planet. And personally, if reasonphrase isn't sufficient I use json-problem to return additional information. tools.ietf.org/html/draft-nottingham-http-problem-04

[Jeffery Thomas]

Sorry, I guess I should have been clearer also. iOS is an area that I've been focusing on lately, so I tend to think of it by default. If you have no expectation of iOS clients, then the limitation of NSHTTPURLResponse is not of any concern.

[Jim Grisham]

See here for a link to another, different, proposal. It refers to draft-cedik-http-warning and the issues discussion at http-warning at GitHub.

[Jim Grisham]

There appears to be a standards-track RFC purpose-made for this sort of thing: RFC 7807 Problem Details for HTTP APIs