Skip to content

Instantly share code, notes, and snippets.

@sandeshbodake
Created May 8, 2024 17:42
Show Gist options
  • Save sandeshbodake/04b45555c480ad62c7bfdba53e5a547a to your computer and use it in GitHub Desktop.
Save sandeshbodake/04b45555c480ad62c7bfdba53e5a547a to your computer and use it in GitHub Desktop.

Revisions

  1. sandeshbodake created this gist May 8, 2024.
    62 changes: 62 additions & 0 deletions sidekiq_batch_process.rb
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,62 @@
    class UpsApiWorker
    include Sidekiq::Job

    sidekiq_options retry: 3 # Retry the job up to 3 times (adjust as needed)

    def perform(zipcode, batch)
    batch.jobs do
    UPS_API_LIMIT.within_limit do
    UpsApiService.fetch(zipcode)
    end
    end
    end

    def success
    send_notification
    end
    end

    private

    def process_zipcode(zipcode)
    # Your job processing logic here
    Sidekiq.logger.info "Processed zipcode #{zipcode}"
    end
    end

    class UpsApiService
    def client
    end

    def fetch
    end
    end

    # Main entry point to enqueue jobs using Sidekiq::Batch
    class JobProcessor
    include Sidekiq::Job

    def self.process(zipcodes)
    Sidekiq::Batch.new.on(:success, UpsApiWorker, 'zipcodes' => runner).jobs do |batch|
    zipcodes.each do |zipcode|
    UpsApiWorker.perform_async(zipcode, batch.bid)
    end
    end
    end
    end

    # Usage example:
    JobProcessor.process(zipcodes)
    ### config for rate limiting
    # 10 requests for 100 seconds
    UPS_API_LIMIT = Sidekiq::Limiter.window('ups_api_limiter', 10, 100)

    Sidekiq::Limiter.configure do |cofig|
    config.errors << UpsApi::Error::TooManyRequests
    end

    Sidekiq.configure_server do |config|
    config.periodic do |mgr|
    mgr.register('0 0 * * FRI', JobProcessor)
    end
    end