S3 REST API and POST method

Question

I'm using AWS S3 REST API, and after solving some annoying problems with signing it seems to work. However, when I use correct REST verb for creating resource, namely POST, I get 405 method not allowed. Same request works fine with method PUT and creates resource.

Am I doing something wrong, or is AWS S3 REST API not fully REST-compliant?

Solution 1

Yes, you are wrong in mapping CRUD to HTTP methods.

Despite the popular usage and widespread misconception, including high-rated answers here on Stack Overflow, POST is not the "correct method for creating resource". The semantics of other methods are determined by the HTTP protocol, but the semantics of POST are determined by the target media type itself. POST is the method used for any operation that isn't standardized by HTTP, so it can be used for creation, but also can be used for updates, or anything else that isn't already done by some other method. For instance, it's wrong to use POST for retrieval, since you have GET standardized for that, but it's fine to use POST for creating a resource when the client can't use PUT for some reason.

In the same way, PUT is not the "correct method for updating resource". PUT is the method used to replace a resource completely, ignoring its current state. You can use PUT for creation if you have the whole representation the server expects, and you can use PUT for update if you provide a full representation, including the parts that you won't change, but it's not correct to use PUT for partial updates, because you're asking for the server to consider the current state of the resource. PATCH is the method to do that.

In informal language, what each method says to the server is:

• POST: take this data and apply it to the resource identified by the given URI, following the rules you documented for the resource media type.

• PUT: replace whatever is identified by the given URI with this data, ignoring whatever is in there already, if anything.

• PATCH: if the resource identified by the given URI still has the same state it had the last time I looked, apply this diff to it.

Notice that create or update isn't mentioned and isn't part of the semantics of those methods. You can create with POST and PUT, but not PATCH, since it depends on a current state. You can update with any of them, but with PATCH you have an update conditional to the state you want to update from, with PUT you update by replacing the whole entity, so it's an idempotent operation, and with POST you ask the server to do it according to predefined rules.

By the way, I don't know if it makes sense to say that an API is or isn't REST-compliant, since REST is an architectural style, not a spec or a standard, but even considering that, very few APIs who claim to be REST are really RESTful, in most cases because they are not hypertext driven. AWS S3 is definitely not RESTful, although where it bears on your question, their usage of HTTP methods follows the HTTP standard most of the time.

Solution 2

+--------------------------------------+---------------------+
|                 POST                 |         PUT         |
+--------------------------------------+---------------------+
| Neither safe nor idempotent Ex: x++; | Idempotent Ex: x=1; |
+--------------------------------------+---------------------+

Solution 3

To add to @Nicholos

From the http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

POST:

The posted entity is subordinate to the URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database

The action performed by the POST method might not result in a resource that can be identified by a URI. In this case, either 200 (OK) or 204 (No Content) is the appropriate response status, depending on whether or not the response includes an entity that describes the result

If a resource has been created on the origin server, the response SHOULD be 201 (Created)

PUT:

The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI. If a new resource is created, the origin server MUST inform the user agent via the 201 (Created) response. If an existing resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to indicate successful completion of the request

IMO PUT can be used to create or modify/replace the enclosed entity.

Solution 4

In the original HTTP specification, the resource given in the payload of a POST request is "considered to be subordinate to the specified object" (i.e. the request URL). TimBL has said previously (can't find the reference) that it was modelled on the identically-named method in NNTP.

View more solutions

Comments

[Charles Engelke]

Good explanation. POST modifies a resource. People get confused because it's common to POST to a resource that represents a collection to modify the collection by creating a new member. But the new member is a different resource, with a different URL, than the one you POSTed to.

[vartec]

Interesting, I did not know that, most sources I've checked talk about mapping CRUD to POST,GET, PUT, DELETE. Although I found some stating that it's POST to /someobjects/ vs PUT to /someobjects/identifier. Which would make sense why S3 API complains, as I'm doing POST to /path/file.name.

[Pedro Werneck]

If you're making POST to /path/file.name attempting to create that resource, it's definitely wrong, since POST data is subordinated to the /path/file.name resource, and if doesn't exist, it can't do anything with it. In theory, AWS should return a 404, not a 405, but that's probably an implementation detail. The route for it has the allowed methods, and it checks the method before checking if the resource actually exists.

[Pedro Werneck]

There are many widespread misconceptions around REST, and that's probably the most common. Unfortunately, Stack Overflow helps to reinforce a lot of them, and good sources are hard to find. It's kind of curious how the most voted answer on the top question with the REST tag in here is absolutely wrong.

[vartec]

en.wikipedia.org/wiki/Restful#RESTful_web_APIs seems to corroborate POST to dir part.

[Pedro Werneck]

That wikipedia article is very misleading and of very poor quality. Almost every single example given is not-RESTful at all, including on the section you referenced, and at least there are some comments on the talking page about this issue. If you need corroboration on anything I said above, you should refer to the RFC 2616.

[Pedro Werneck]

@k0pernikus RFC 2616 is now obsolete. Please, refer to RFC 7231. tools.ietf.org/html/rfc7231

[kolobok]

"POST is the method used for any operation that isn't standardized by HTTP" - I don't see this in RFC spec. Where did you get this?

[Pedro Werneck]

@kolobok Roy Fielding, roy.gbiv.com/untangled/2009/it-is-okay-to-use-post. Also, that's in the RFC, but you won't find a literal quote there.

[HalfWebDev]

Solid answer ! Just a question. Which is the correct way of creating data?

[cowlinator]

In as far as Update maps to PUT/POST/PATCH, and Create maps to PUT/POST, it sound to me like you are describing HTTP, not REST. As I understand it, REST has artificially and purposefully mapped CRUD to POST, GET, PUT, DELETE, in spite of those methods' HTTP definitions. See en.wikipedia.org/wiki/…

[Pedro Werneck]

@cowlinator The Wikipedia article is wrong. REST can't change the semantics of the underlying protocol. If you're doing that, it's not REST.

[cowlinator]

@PedroWerneck, Sorry for my lack of understanding in this area, I am not a web developer. After reading into it, I think you are right... However, shouldn't it still be possible to map HTTP methods to at least CRD (Create/Read/Delete), with PUT/GET/DELETE respectively?

[Pedro Werneck]

@cowlinator You can do whatever you want with the methods, but you can't call if REST if you're not following the protocol semantics as described in the RFC.

[freddieRV]

"if the resource identified by the given URI still has the same state it had the last time I looked..." so you either expect the API to remember the state of the given resource, which makes the result of the request dependent on a previous request, thus not following REST at all, or you have to send the previous status along the new one for the API to compare and make the necessary changes, which doesn't sound quite correct to me either.

[Pedro Werneck]

@freddieRV No. That's exactly what the constraint of statelessness means. Every request must have all the data needed to determine its results, and not depend on any shared context on the server. If that means sending a previous known state to validate changes against, no problem with that.

[RAM]

The comment regarding POST Is incorrect. Taken from RFC 7231, which was published in 2014. In section 4.3.3 " For example, POST is used for the following functions (...) Creating a new resource that has yet to be identified by the origin server" RFC 7231 just clarifies what was always POST's use case.

[Pedro Werneck]

@RAM Read the first paragraph: The POST method requests that the target resource process the representation enclosed in the request according to the resource's own specific semantics.

[Pedro Werneck]

@RAM That's one example, one use case. The description encompasses that example, but also other use cases.

[Pedro Werneck]

@RAM Please, pay attention to the question I was answering and to what you're saying. If someone is using POST for creation, the target resource can't be the very same resource being created. The identified target is responsible for creating the yet to be identified new resource. That's just simple logic and anyone who implemented the factory pattern understands it.