I'm currently running ActiveJob with DelayedJob as the backend for my background jobs on Heroku with 10 worker dynos. Daily, I need to run ~2000+ jobs which require lots of interaction with Google Sheets API that can take ~60+ seconds after the worksheet has run its calculations.
As each job could potentially take more than a minute to run, I'm wondering how I can increase the efficiency of these workers? It seems to me that these 10 workers can only take on 1 task at a time. Is it possible for 1 worker to take on my jobs? Would switching my background service to Sidekiq or another service allow these workers to take on more jobs?
Any insight would be appreciated, thanks!