Increasing concurrency in Redash 9.0

Support Usage Support
#1

Hi,
I was testing out the new redash version. One thing I struggled initially with celery redash was concurrency which after some reading I was able to increase by having multiple workers and each worker having concurrency=4.

I am not that familiar with rabbitmq and would like to know how to increase concurrency. eg - I want 24 people to run their queries at the same time. Do I just need to increase the workers or is there another setting I have to use? Also for celery I had launch multiple worker docker containers. Is the process same for RQ or can I just launch one container with multiple workers?

It would also help if it could be added as documentation in the docs or in the release notes.

#2

Just a note for readers, at present Redash v9 is in beta so the documentation has not been updated yet.

#3

That’s fine!
Can you help with the concurrency in Rq with redash?

#4

I would also definitely be interested in knowing this

Love, The beta v9 … looks cleaner and the tooltips feature is really useful

#5

+1 on understanding the best practice of concurrency in RQ with redash.

#6

Hi Shitij/all,

Could you please explain how you ramped up your workers and concurrency in v8 to 4. I am struggling with that bit.

For example here is my code for scheduled_worker,

If i wanted to increase the worker count to 4 and concurrency to 4 in below code. Do i just increase my WORKERS_COUNT=4. what flag would i need to add for the concurrency part ?

docker container run -d --read-only --restart always --name scheduled_worker --network redash_default --tmpfs=/tmp -e QUEUES=scheduled_queries,schemas -e WORKERS_COUNT=1 --network-alias scheduled_worker --env-file env redash/redash:8.0.0.b32245 worker