Delete multiple records using REST

Question

What is the REST-ful way of deleting multiple items?

My use case is that I have a Backbone Collection wherein I need to be able to delete multiple items at once. The options seem to be:

1. Send a DELETE request for every single record (which seems like a bad idea if there are potentially dozens of items);
2. Send a DELETE where the ID's to delete are strung together in the URL (i.e., "/records/1;2;3");
3. In a non-REST way, send a custom JSON object containing the ID's marked for deletion.

All options are less than ideal.

This seems like a gray area of the REST convention.

Solution 1

1. Is a viable RESTful choice, but obviously has the limitations you have described.
2. Don't do this. It would be construed by intermediaries as meaning “DELETE the (single) resource at /records/1;2;3” — So a 2xx response to this may cause them to purge their cache of /records/1;2;3; not purge /records/1, /records/2 or /records/3; proxy a 410 response for /records/1;2;3, or other things that don't make sense from your point of view.
3. This choice is best, and can be done RESTfully. If you are creating an API and you want to allow mass changes to resources, you can use REST to do it, but exactly how is not immediately obvious to many. One method is to create a ‘change request’ resource (e.g. by POSTing a body such as records=[1,2,3] to /delete-requests) and poll the created resource (specified by the Location header of the response) to find out if your request has been accepted, rejected, is in progress or has completed. This is useful for long-running operations. Another way is to send a PATCH request to the list resource, /records, the body of which contains a list of resources and actions to perform on those resources (in whatever format you want to support). This is useful for quick operations where the response code for the request can indicate the outcome of the operation.

Everything can be achieved whilst keeping within the constraints of REST, and usually the answer is to make the "problem" into a resource, and give it a URL.
So, batch operations, such as delete here, or POSTing multiple items to a list, or making the same edit to a swathe of resources, can all be handled by creating a "batch operations" list and POSTing your new operation to it.

Don't forget, REST isn't the only way to solve any problem. “REST” is just an architectural style and you don't have to adhere to it (but you lose certain benefits of the internet if you don't). I suggest you look down this list of HTTP API architectures and pick the one that suits you. Just make yourself aware of what you lose out on if you choose another architecture, and make an informed decision based on your use case.

There are some bad answers to this question on Patterns for handling batch operations in REST web services? which have far too many upvotes, but ought to be read too.

Solution 2

If GET /records?filteringCriteria returns array of all records matching the criteria, then DELETE /records?filteringCriteria could delete all such records.

In this case the answer to your question would be DELETE /records?id=1&id=2&id=3.

Solution 3

I think Mozilla Storage Service SyncStorage API v1.5 is a good way to delete multiple records using REST.

Deletes an entire collection.

DELETE https://<endpoint-url>/storage/<collection>

Deletes multiple BSOs from a collection with a single request.

DELETE https://<endpoint-url>/storage/<collection>?ids=<ids>

ids: deletes BSOs from the collection whose ids that are in the provided comma-separated list. A maximum of 100 ids may be provided.

Deletes the BSO at the given location.

DELETE https://<endpoint-url>/storage/<collection>/<id>

http://moz-services-docs.readthedocs.io/en/latest/storage/apis-1.5.html#api-instructions

Solution 4

This seems like a gray area of the REST convention.

Yes, so far I have only come accross one REST API design guide that mentions batch operations (such as a batch delete): the google api design guide.

This guide mentions the creation of "custom" methods that can be associated via a resource by using a colon, e.g. https://service.name/v1/some/resource/name:customVerb, it also explicitly mentions batch operations as use case:

A custom method can be associated with a resource, a collection, or a service. It may take an arbitrary request and return an arbitrary response, and also supports streaming request and response. [...] Custom methods should use HTTP POST verb since it has the most flexible semantics [...] For performance critical methods, it may be useful to provide custom batch methods to reduce per-request overhead.

So you could do the following according to google's api guide:

POST /api/path/to/your/collection:batchDelete

...to delete a bunch of items of your collection resource.

Solution 5

I've allowed for a wholesale replacement of a collection, e.g. PUT ~/people/123/shoes where the body is the entire collection representation.

This works for small child collections of items where the client wants to review a the items and prune-out some and add some others in and then update the server. They could PUT an empty collection to delete all.

This would mean GET ~/people/123/shoes/9 would still remain in cache even though a PUT deleted it, but that's just a caching issue and would be a problem if some other person deleted the shoe.

My data/systems APIs always use ETags as opposed to expiry times so the server is hit on each request, and I require correct version/concurrency headers to mutate the data. For APIs that are read-only and view/report aligned, I do use expiry times to reduce hits on origin, e.g. a leaderboard can be good for 10 mins.

For much larger collections, like ~/people, I tend not to need multiple delete, the use-case tends not to naturally arise and so single DELETE works fine.

In future, and from experience with building REST APIs and hitting the same issues and requirements, like audit, I'd be inclined to use only GET and POST verbs and design around events, e.g. POST a change of address event, though I suspect that'll come with its own set of problems :)

I'd also allow front-end devs to build their own APIs that consume stricter back-end APIs since there's often practical, valid client-side reasons why they don't like strict "Fielding zealot" REST API designs, and for productivity and cache layering reasons.

View more solutions

Comments

[Luka Žitnik]

Possible duplicate of Restful way for deleting a bunch of items

[Etzeitet]

Depending on the format of your url/query string your server should be able to determine whether DELETE /records/1;2;3 or whatever is a single or multiple delete event. DELETE /records?id=1;2;3;4&multi=1 for example?

[Nicholas Shanks]

It's not your server that you have to worry about, it's intermediaries, CDNs, caching proxies, etc. The internet is a layered system. That is the reason it works so well. Roy determined which aspects of the system were necessary for its success, and named them REST. If you issue a DELETE request, whatever lies between the requestee and the server will think a single resource, at the specified URL, is being deleted. Query strings are opaque parts of the URL to these devices, so it doesn't matter how you specify your API, they are not privy to this knowledge so cannot behave differently.

[Luke Puplett]

I also came to this conclusion: just flip the verb to what you want to do. I don't understand how what goes for GET does not go for DELETE.

[Nicholas Shanks]

GET /records?id=1&id=2&id=3does not mean “get the three records with IDs 1, 2 & 3”, it means “get the single resource with URL path /records?id=1&id=2&id=3” which might be a picture of a turnip, a plain text document containing the number "42" in chinese, or may not exist.

[Nicholas Shanks]

Consider the following: two sequential requests for /records?id=1 and /records?id=2 are sent, and their responses cached by some intermediary (e.g. your browser or ISP). If the internet knew what your application meant by this, then it stands to reason that a request for /records?id=1&id=2 could be returned by the cache simply by merging (somehow) the two results it already has, without having to ask the origin server. But this is not possible. /records?id=1&id=2 might be invalid (only 1 ID allowed per request) or may return something completely different (a turnip).

[dukethrash]

/records/1;2;3 will not work if you have a lot resources to delete due to URI length restrictions

[Luke Puplett]

This is a basic resource caching problem. If my DBA mutated the state directly, then caches are now out of sync. You give an example 410 returned by the intermediary, but 410 is for permanent removals, upon DELETE a cache might clear its slot for that URL, but it won't send a 410 or a 404, since it doesn't know if a DBA hasn't just put the resource immediately back again at origin.

[Luke Puplett]

Note that if considering DELETE and a body defining the resources to purge, that some intermediaries may not forward the body. Also, some HTTP clients cannot add a body to a DELETE. See stackoverflow.com/questions/299628/…

[Nicholas Shanks]

@LukePuplett I would simply state that passing a request body with a DELETE request is forbidden. Don't do it. If you do I will eat your children. Nom nom nom.

[Luke Puplett]

. o O ( he knows I have children )

[Kevin Meredith]

stackoverflow.com/a/299696/409976 disagrees with your "passing a request body with a DELETE request is forbidden," @NicholasShanks

[Nicholas Shanks]

@KevinMeredith Hi, and thanks for commenting. Per spec, yes it's not forbidden (though note shelly's comment on that answer); but Luke was suggesting how the OP might define his API. I merely suggested that my API would state that sending it a DELETE request with a body was forbidden.

[LB2]

The problem with argument for #3 is that it carries the same penalty as counter argument against #2. Creating to-delete resource is not something that upstream proxies will know how to handle - the same counter argument that is raised against approach #2.

[Nicholas Shanks]

@LB2 I understand where you're coming from, and you're right, middle-men won't know that creating of this resource will result in other resources going away; but other resources go away all the time. What's key is that the guys in the middle don't do the wrong thing in this case, even if what they do do is not optimal.

[Luka Žitnik]

PATCH is meant to change a specific resource -- restcookbook.com/HTTP%20Methods/patch, restful-api-design.readthedocs.io/en/latest/methods.html

[Nicholas Shanks]

@user152807 Yes, and in this instance it is changing the specific resource of "the list of records". Do not forget that a collection is a resource in and of itself, and its content is its list of members. Note though that removing an item from a collection does not necessarily have to delete the referenced resource, but that it could.

[Luka Žitnik]

@NicholasShanks sorry, I didn't read that correctly, it said "send a PATCH request to the list resource" in bold.

[Joseph Nields]

@NicholasShanks I really disagree. If the results are cached, that's the fault of the server. And if you're talking about the design of the API, you're hopefully the one writing the code for the server. Whether you use id[]=1&id[]=2 or id=1&id=2 in the query string to represent an array of values, that query string does represent just that. And I think it is extremely common & good practice to have the query string represent a filter. Besides, if you allow for deletes and updates, don't cache the GET requests. If you do, clients will hold stale state.

[Nicholas Shanks]

@JosephNields you have misunderstood me, our two points are not incompatible. I am saying a resource can represent anything the server wants it to, including a collection of two items, but intermediaries won't understand that /?ids=1,2 produces a collection of /1 and /2 — this knowledge is internal to the origin server. Even clients should not know this, if you're using hypertext RESTfully. When I said "does not mean" in my original comment, I was referring to "from HTTP's point of view", not from the server's.

[Nicholas Shanks]

I loved this answer right up until I read the last sentence :) I have never seen a use case where applying strict REST has had a net detrimental effect. Sure it can make for more code to write at both ends, but you end up with a safer, cleaner, less coupled system.

[Luke Puplett]

Haha. It's actually become a pattern! Backend for front-end it's called on ThoughtWorks technology radar. It also allows more application logic to be written that would be cumbersome in say, JavaScript, and obviously can be updated without a client so update say, for an iOS app.

[Nicholas Shanks]

Skim-reading the first four hits from Google, it seems that this BFF technique can only work when the clients are under your control. The client developers develop the API they want, mapping calls to microservice APIs that are the real back end. In this diagram: samnewman.io/patterns/architectural/bff/#bff I would place the "Perimeter" line below the BFF boxes — each box is simply part of the client. It could even live outside of the data centre housing the microservices. I also don't see how REST doesn't apply to both interfaces (client/BFF and BFF/microservice).

[Luke Puplett]

Yeah that's a good point. It's usually for when you have a team building microservices and a team making an angular app, for example, and that dev team are more front-end types who don't like having to work against a bunch of little purist services. Though I don't see any reason you can't use the same pattern for abstracting microservices and aggregation into a more usable facade for your customers, such that the microservices can be changed without impacting the facade.

[Joseph Nields]

@NicholasShanks, fair, it's true this is internal to the server, but isn't a RESTful API just an API? We have to define behavior & how clients interface with it. GET api/records/1 could return a picture of a turnip, but if we say it should return a JSON representation of a "record" object, clients can expect that. A DELETE request with a query string doesn't have to represent deleting the records that match a filter, but that doesn't mean that it can't / shouldn't. This approach achieve the desired result without breaking RESTful constraints, so I think it is a good solution.

[Nicholas Shanks]

@JosephNields I agree with all of that :-). The other downside to having a list of IDs in your URL though is that URI length is more limited in the real world than body length, which can be a problem if you want to list half a million IDs.

[Joseph Nields]

@NicholasShanks you're right, and I've dealt with that problem first hand before. Some people configure the server to accept the x-method-override header and instead POST the request with x-www-form-urlencoded if this issue can arise. Just a couple hundred ids could cause issues!

[user1633272]

The option 3 have the same problem as option 2, isn't it? I mean intermediaries not purge /records/1, /records/2 or /records/3.

[Nicholas Shanks]

@user1633272 You're right — it's a compromise, but one that doesn't violate REST. Even if an intermediary did purge that URL or returned cached results for now-deleted URLs, nothing bad will happen, other than clients getting stale data (all data is stale by the time it reaches the client anyway).

[Abhishek Sharma]

I think the primary problem with this is that any non-idempotent request (specially delete and update) should be carefully handled when it comes to running queries on multiple records at the same time; mainly due to unprecedented edge cases and errors.

[Victorio Berra]

An API endpoint should model the needs of the domain and business. Code to solve those problems and avoid over-engineering to adhere to strict and many time's inflexible specs. REST is nothing but guidelines anyway.

[Ravik]

Link to "HTTP API architectures" is no longer working.

[Nicholas Shanks]

@Ravik thanks – fixed. It's moved at least 3 times since I wrote this answer!

[tempcke]

This seems like a good solution. I guess if mozilla thinks it is correct then it must be? The only question then is error handling. Suppose they pass ?ids=1,2,3 and id 3 does not exist do you delete 1 and 2 then respond with a 200 because the requester wants 3 gone and it is not there so it does not matter? or what if they are authorized to delete 1 but not 2 ... do you delete nothing and respond with an error or do you delete what you can and leave the others...

[tonnoz]

I would like to suggest this article: «REST was never about CRUD» tyk.io/blog/rest-never-crud . It helped me a lot understanding REST better. It deals the BULK/BATCH resource problem but also much more (e.g. long running batches, transactional resources for long running tasks etc..). Give it a shot

[Daniele]

Is a viable solution that the list of items is communicated via a JSON formatted array?

[Felix K.]

yes sure. you can POST a payload in which the ids are sent via a json array.

[鄭元傑]

It is interesting that Google API guide said If the HTTP verb used for the custom method does not accept an HTTP request body (GET, DELETE), the HTTP configuration of such method must not use the body clause at all, at Custom Method chapter. But the GET accounts.locations.batchGet api is GET method with body. That's is weird. developers.google.com/my-business/reference/rest/v4/…

[Felix K.]

@鄭元傑 agree, looks a bit weird on the first sight but if you look closely it is actually a POST http method used and only the custom method is named batchGet. I guess google does it to (a) stick with their rule that all custom methods need to be POST (see my answer) and (b) to make it easier for people to put a "filter" in the body so you do not have to escape or encode the filter as with query strings. the downside, of course, is that this is not really cacheable anymore...

[deamon]

https://service.name/v1/some/resource/name:customVerb is not RESTful by definition.

[SleepNot]

If passing multiple items to the body of the request - shouldn't it be an PUT rather than DELETE?

[Nicholas Shanks]

@SleepNot Yes. I don't know if you were referring to a comment or to my answer, but I do say to use POST if using option #3.

[Nathan Phetteplace]

I typically will return a successful response because the end state is the same regardless. This simplifies logic on the client as well since they no longer have to handle that error state. As for the authorization case, I would just fail the whole request...but really it depends on your use case.

[Nicholas Shanks]

I concur with Nathan.

[Dave]

@deamon - Why not? The custom operation IS the resource in this case. And you are POSTing to it which, as the Google API Guide author pointed out, has the most flexible semantics. While ReST APIs often look like thin CRUD wrappers, this isn't a requirement.

[Erik Philips]

stackoverflow.com/questions/299628/….

[Lam Duc Trung]

What are you trying to say?