Wait for all Operations in queue to finish before performing task

Question

I have an Operation subclass and Operation queue with maxConcurrentOperationCount = 1.

This performs my operations in a sequential order that i add them which is good but now i need to wait until all operations have finished before running another process.

i was trying to use notification group but as this is run in a for loop as soon as the operations have been added to the queue the notification group fires.. How do i wait for all operations to leave the queue before running another process?

for (index, _) in  self.packArray.enumerated() {

    myGroup.enter()
    let myArrayOperation = ArrayOperation(collection: self.outerCollectionView, id: self.packArray[index].id, count: index)
    myArrayOperation.name = self.packArray[index].id
    downloadQueue.addOperation(myArrayOperation)
    myGroup.leave()

}

myGroup.notify(queue: .main) {
 // do stuff here
}

Solution 1

You can use operation dependencies to initiate some operation upon the completion of a series of other operations:

let queue = OperationQueue()

let completionOperation = BlockOperation {
    // all done
}

for object in objects {
    let operation = ...
    completionOperation.addDependency(operation)
    queue.addOperation(operation)
}

OperationQueue.main.addOperation(completionOperation)  // or, if you don't need it on main queue, just `queue.addOperation(completionOperation)`

Or, in iOS 13 and later, you can use barriers:

let queue = OperationQueue()

for object in objects {
    queue.addOperation(...)
}

queue.addBarrierBlock {
    DispatchQueue.main.async {
        // all done
    }
}

Solution 2

A suitable solution is KVO

First before the loop add the observer (assuming queue is the OperationQueue instance)

queue.addObserver(self, forKeyPath:"operations", options:.new, context:nil)

Then implement

override func observeValue(forKeyPath keyPath: String?, of object: Any?, change: [NSKeyValueChangeKey : Any]?, context: UnsafeMutableRawPointer?) {
    if object as? OperationQueue == queue && keyPath == "operations" {
        if queue.operations.isEmpty {
            // Do something here when your queue has completed
            self.queue.removeObserver(self, forKeyPath:"operations")
        }
    } else {
        super.observeValue(forKeyPath: keyPath, of: object, change: change, context: context)
    }
}

Edit:

In Swift 4 it's much easier

Declare a property:

var observation : NSKeyValueObservation?

and create the observer

observation = queue.observe(\.operationCount, options: [.new]) { [unowned self] (queue, change) in
    if change.newValue! == 0 {
        // Do something here when your queue has completed
        self.observation = nil
    }
}

Since iOS13 and macOS15 operationCount is deprecated. The replacement is to observe progress.completedUnitCount.

Another modern way is to use the KVO publisher of Combine

var cancellable: AnyCancellable?

cancellable = queue.publisher(for: \.progress.completedUnitCount)
    .filter{$0 == queue.progress.totalUnitCount}
    .sink() { _ in 
       print("queue finished") 
       self.cancellable = nil           
    }

Solution 3

I use the next solution:

private let queue = OperationQueue()

private func addOperations(_ operations: [Operation], completionHandler: @escaping () -> ()) {
    DispatchQueue.global().async { [unowned self] in
        self.queue.addOperations(operations, waitUntilFinished: true)
        DispatchQueue.main.async(execute: completionHandler)
    }
}

Solution 4

Set the maximum number of concurrent operations to 1

operationQueue.maxConcurrentOperationCount = 1

then each operation will be executed in order (as if each was dependent on the previous one) and your completion operation will execute at the end.

View more solutions

Comments

[WanderingScouse]

Very clean. Thanks

[Pavan]

Hey Rob, I've done exactly this but my completionOperation is still being prematurely fired. My operations are asynchronous network operations so an operation is only considered finished when the response comes back. I read that I need to override the isAsynchronous property among a few others to make it an asynchronous operation, but I read that the property will be ignored if I add the operation to a queue anyway. So I'm confused on what to do. Could you please advise further?

[Siempay]

what if there is an array of operations in a loop and I wanna wait for all of theme

[Rob]

I don’t know what you mean. That’s what the above does, waits to fire the completion operation until they’re all done.

[Rob]

Re isAsynchronous: It is true that isAsynchronous is ignored when adding to a queue, but all of the isExecuting, isFinished and the associated KVO is required when adding operations that wrap some asynchronous process to a queue. That having been said, I believe it is best practice to set isAsynchronous to true when the operation is, indeed, asynchronous, to (a) reflect the reality of the operation and (b) to ensure it works whether it is used in a queue or just manually started.

[hgwhittle]

Regarding the operation dependencies technique here, it looks like main is a class var on OperationQueue? Should that line be changed to OperationQueue.main.addOperation(completionOperation)? Does that have the same implications?

[Rob]

@hgwhittle good catch. Both approaches work for the completion operation pattern (but barrier approach only works on the designated queue). Adjusted answer accordingly.

[Left as an exercise]

Just be careful here: this only holds if all operations in the queue have the same relative queuePriority and become ready in the order they were added to the serial queue. See the "Determine the Execution Order" section in the OperationQueue docs here

[tospig]

the operationCount property was deprecated. Does that mean it shouldn't be used?