Redash V10 decoupled redis

Support Self Hosted Redash Support
1

Issue Summary

Hello,

I upgraded redash from V7 (legacy supervisor setup) to V10 (docker based setup). We are running redash on a larger scale where we have setup 100s of workers under queries and scheduled queries queues. We are using decoupled redis server and postgres meta data store.
Everything was smooth with V7. We ran V7 almost 3 years without major issues.

With V10 we are facing below issues

  1. Queries are getting stuck in queue
  2. Redash admin page sometimes doesn’t load
  3. There are sometimes 1000s of jobs under default queue

My first guess is that new setup is not using correct redis server. below is part of docker file

redis:
image: redis:5.0-alpine
restart: always
postgres:
image: postgres:9.6-alpine
env_file: /opt/redash/env
volumes:
- /opt/redash/postgres-data:/var/lib/postgresql/data
restart: always
nginx:
image: redash/nginx:latest

My env file contains redis cluster variable. How can I know if docker file is using that cluster?

Do I need postgres and redis section in my docker compose?
What other things I should check?

thanks

Technical details:

  • Redash Version: V10 (latest)
  • Browser/OS:
  • How did you install Redash:
2

Hi there, awesome to hear that Redash is being used at such a scale!

I agree with your analysis: it looks like there is a mismatch between where jobs are being queued and where they’re being retrieved. I’m guessing your queues are growing endlessly until the job TTL expires.

The specific environment variable to use is REDASH_REDIS_URL. You can check the value for any given docker service by running docker-compose exec bash and then echo $REDASH_REDIS_URL.

Nope.