Home python flask run a slow function off a webhook that requires a response in 5 seconds
Reply: 1

python flask run a slow function off a webhook that requires a response in 5 seconds

Harlekuin Published in 2018-02-14 06:14:02Z

This question already has an answer here:

  • How do I run a long-running job in the background in Python 4 answers

Running on Windows.

This is similiar to the question python flask - run script after processing the request however the answers to that one were insufficient.

Given the code:

app = Flask(__name__)

def processWebhook():
    time.sleep(30) # slow_function()
    return "received"

if __name__ == '__main__':

I need the slow_function to run for each POST, however it can't stop the response being returned within 5 seconds (say the function is time.sleep(30)).

Also, each slow_function cannot start until the one before has finished.

Is there a way I can add "jobs" to a second thread that execute linearly rather than just calling slow_function?

Christian Sauer
Christian Sauer Reply to 2018-02-14 06:25:04Z

You simply cannot run long running processes in a web server, they are build to do the opposite and trying to force them to do it leads to ruin in the end.

Your best bet is to offload the long running function to a separate python process. Have a look at celery: http://www.celeryproject.org/ You do something homegrown using Redis / RabbitMq and consuming program.

Your requirement that only one function runs at a time should be achievable with all three.

Note that the number of moving parts increases if you use this solution, instead of one process, you have at least three (webserver, redis /rabbitmq and your consumer). Since orchestrating that is annoying, you might want to use docker (and docker-compose) to simplify the orchestration, e.g. restarting a component or monitoring it.

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.342562 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO