addprocs_sge from ClusterManagers.jl relies on the blocking
addprocs from Distributed.jl. Therefore, every time I request a node, I must wait until the node is acquired before proceeding.
So, while the following code snippet immediately returns, for the second qsub command to be submitted, it must wait for the first qsub command to start running.
@async is effectively useless. The simplest solution I found to overcome this is as follows:
sge = SGEManager(1, "");
id = Distributed.addprocs_locked(sge; qsub_env="", res_list="");
a = 
@async push!(a, get_proc())
I do this because,
addprocs is just a locked wrapper over
function addprocs(manager::ClusterManager; kwargs...)
My question is: what are potential issues that may arise when this happens? Why is there a lock in the first place? Finally, is there any better way to do this?
I am aware of this thread but nothing really came out of it apparently.