Disable automatic retry with ActiveJob, used with Sidekiq

Question

Is there a way to disable automatic retry with ActiveJob and Sidekiq ?

I know that with Sidekiq only, we just have to put

sidekiq_options :retry => false

as mentioned here : https://github.com/mperham/sidekiq/wiki/Error-Handling#configuration

but it doesn't seem to work with ActiveJob and Sidekiq.

I also know the solution to entierly disable the retry as proposed here : https://stackoverflow.com/a/28216822/2431728

But it's not the behavior I need.

Solution 1

Ok thanks for the answer.

Just for information, I also asked the question in an issue related to this subject on ActiveJob Github repository : https://github.com/rails/activejob/issues/47

DHH answered me a solution I haven't tested but that can do the job.

Personnally, I finally put this in an initializer in order to disable Sidekiq retries globally and it works well :

Sidekiq.configure_server do |config|
   config.server_middleware do |chain|
      chain.add Sidekiq::Middleware::Server::RetryJobs, :max_retries => 0
   end
end

Solution 2

As of sidekiq 6.0.1, it is possible to pass the following to an ActiveJob worker to prevent it from retrying:

class ExampleJob < ActiveJob::Base
  sidekiq_options retry: false

  def perform(*args)
    # Perform Job
  end
end

More information: https://github.com/mperham/sidekiq/wiki/Active-Job#customizing-error-handling

EDIT:

According to this this requires Rails 6.0.1 or later as well.

Solution 3

There is no way to configure anything about Sidekiq with ActiveJob. Use a Sidekiq Worker if you don't want to use the defaults.

Solution 4

You can catch up the exception and to do nothing instead retry or to configure retry:

  class ExampleJob < ActiveJob::Base
    rescue_from(StandardError) do |exception|
      Rails.logger.error "[#{self.class.name}] Hey, something was wrong with you job #{exception.to_s}"       
    end

    def perform
      raise StandardError, "error_message"
    end
  end

  class ExampleJob < ActiveJob::Base
    rescue_from(StandardError) do |exception|
      retry_job wait: 5.minutes, queue: :low_priority     
    end

    def perform
      raise StandardError, "error_message"
    end
  end

For running retrying you can use retry_on method retry_on method doc

Sidekiq wiki for retries with Active Job integration

Solution 5

I had this same need, ie ActiveJob wrapping Sidekiq but wanting to support max_retries. I put this in an initializer. If #max_retries is defined on an ActiveJob job, it will be used to set retries. If #ephemeral? is defined and returns true, job will not be rerun and will not be transferred to 'dead' if it fails.

class Foobar::SidekiqClientMiddleware
  def call(worker_class, msg, queue, redis_pool)
    aj_job = ActiveJob::Base.deserialize(msg['args'][0]) rescue nil
    msg['retry'] = aj_job.respond_to?(:max_retries) ? aj_job.max_retries : 5
    msg['retry'] = false if aj_job.respond_to?(:ephemeral?) && aj_job.ephemeral?
    yield
  end
end

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{redis_host}:6379/12" }
  config.client_middleware do |chain|
    chain.add Foobar::SidekiqClientMiddleware
  end
end

Sidekiq.configure_server do |config|
  config.redis = { url: "redis://#{redis_host}:6379/12" }
  config.client_middleware do |chain|
    chain.add Foobar::SidekiqClientMiddleware
  end
end

Note: it actually is important to add this to the middleware chain for both client and server if any of your jobs create new jobs themselves as they are executed.

View more solutions

Comments

[gerry3]

actually, you can remove the RetryJobs middleware as shown here

[Pratik Bothra]

Even if we use an initializer, and the settings are as follows? Sidekiq.default_worker_options = { 'backtrace' => 5, 'retry' => 3 }

[Ari]

Sidekiq has a builtin way to turn off retries globally: Sidekiq.default_worker_options = { retry: 0 }

[courtsimas]

@Ari I don't believe that works for ActiveJob though... only for native Sidekiq workers without AJ

[Foton]

@courtsmas Sidekiq::Middleware::Server::RetryJobs was removed in 5.0.0 so it seems that only way to disable now is Sidekiq.options[:max_retries] = 0.