Including the .env file in the docker-compose.yml

Support Self Hosted Redash Support
#1

The default setup described here:

Includes an .env file that defines the following variables:

PYTHONUNBUFFERED=0
REDASH_LOG_LEVEL=INFO
REDASH_REDIS_URL=redis://redis:6379/0
POSTGRES_PASSWORD= X
REDASH_COOKIE_SECRET= X
REDASH_SECRET_KEY= X
REDASH_DATABASE_URL=postgresql://postgres:ft9wZJhdyL7hnKQNdYGOxMWufYsvmshq@postgres/postgres

I’m trying to move my redash stack over to my Qnap NAS. In order to do this my docker-compose.yml can not (as far as I know) have any references to external files, such as the .env file*.

Is there some way to write in the enviroment variables directly in the docker-compose-yml to bypass this?

  • This because as opposed to in e.g. Linux, you do not make a folder and place you docker-compose-yml in it along with setup files such as .env. In Qnap you just run the yml configuration through the system’s own Container Station.
#2 
version: "2"
x-redash-service: &redash-service
  image: redash/redash:8.0.0.b32245
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always
services:
  server:
    <<: *redash-service
    command: server
    ports:
      - "5000:5000"
    environment:
      REDASH_WEB_WORKERS: 4
  scheduler:
    <<: *redash-service
    command: scheduler
    environment:
      QUEUES: "celery"
      WORKERS_COUNT: 1
  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries,schemas"
      WORKERS_COUNT: 1
  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2
  redis:
    image: redis:5.0-alpine
    restart: always
  postgres:
    image: postgres:9.6-alpine
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    restart: always
  nginx:
    image: redash/nginx:latest
    ports:
      - "80:80"
    depends_on:
      - server
    links:
      - server:redash
    restart: always

The standard docker-compose.yml includes the .env file in multiple locations. What I am trying to figure out how to do is to get rid of those references.

#3

I think you can achieve this by adding an environment token to your docker-compose.yaml file. More info here.

#4

Thanks! I copy pasted the enviroment variables as such, but now I ran into another problem:

x-redash-service: &redash-service
  image: redash/redash:8.0.0.b32245
  depends_on:
    - postgres
    - redis
  environment:
    PYTHONUNBUFFERED: 0
    REDASH_LOG_LEVEL: "INFO"
    REDASH_REDIS_URL: "redis://redis:6379/0"

The REDASH_REDIS_URL is setup like this now. But when I run docker-compose up:

[2020-12-29 17:02:31,921][PID:1][ERROR][MainProcess] consumer: Cannot connect to redis://localhost:6379/0: Error 99 connecting to localhost:6379. Cannot assign requested address…

It seems the “redis://redis:6379/0” is somehow converted to “redis://localhost:6379/0”, and this is the source of the problem. Is there some way to fix this so that redish stays as redis?

#5

I think this doesn’t work because your environment variables are declared before your services. Since redis doesn’t exist yet it is converted to localhost. You could probably work around this another way by using the extends keyword for each of your Redash services. Just make each of them extend a service with the environment declared. Docs here.

But…

Looking back at your original question though I noticed this:

Have you tested this? Because the docker-compose docs make it pretty clear that .env files will be honored.

Compose supports declaring default environment variables in an environment file named .env placed in the folder where the docker-compose command is executed (current working directory) .

The way this wouldn’t work is if your QNAP NAS is running a modified version of docker-compose that specifically excludes it.

What happens when you try running with a .env file?

#6

image

Like this. You can put .yml and .env in same path.

#7

Thanks!

The problem is that in QNAP I have no idea where the docker-compose.yml file is being run, as opposed to Linux you just run it through the GUI and there is no options about location (as far as I have been able to see). so I have no way of placing the .env file somewhere relative to it.

To be clear, they way Linux and Qnap differs is:

Linux

  • Put .env file and docker-compose.yml in same folder
  • docker-compose up
  • You know in which folder both are

Qnap

  • GUI where you can enter docker-compose instructions, nothing else
  • Press create
  • Its running somewhere, you don’t know where.

Hence I’m trying to figure out how to do everything from the docker-compose, without any dependent files.

#8

Don’t let this stop you :wink: It’s still running docker-compose under the covers which means there is a directory you can store an .env file. I searched google and this was the top result:

And here’s the magic:

You may already know Container Station creates a shared folder named “Container” when it is installed. Inside this folder is another folder with special permissions that contains all the data files for Container Station to run + any container images.

So you can just drop your .env file into that folder on the QNAP and it will work.

#9

Thank you!

Yeah, easiest way seems to be to SSH in to the QNAP and then you can controll it with regular command line interface. i.e. create folder, throw the yml and env file in there and the docker-compose up.

Thanks for setting me on the right path! :slight_smile:

1 Like
#10

Let us know if you get this working. It could be helpful to other Redashers down the line.

#11

It’s working but I’m running in to the (unrelated as far as I know) issue described in this thread here as I do docker-compose up. Not sure what’s going on, apparently the database is not populated successfully. The solution can also be found here:

Shot answer, yes its working. Here’s the steps: :slight_smile:

#1. Container Station seems to put container station data by default to the address below, so that’s where I made a “redash” folder:

Container/container-station-data/application/redash/

#2. Throw the docker-compose.yml + .env file to the folder. I named my env file redash.env, seems to be easier to refer to it in the yml file if it has a name.

#3. SSH in to the QNAP.

ssh XXX.XXX.X.XXX -l admin
(You will be prompted for password. I think QNAP only accepts SSH connection from “admin”)

#4.
cd /share/Container/container-station-data/application/redash/
(QNAP puts the “share” in the beginning for some reason)

#5. If everything is ready:
docker-compose up

If you want to do it in detached mode add the “-d” flag. I think its easier to see the logs without -d.

#6. If something is wrong
docker-compose down
Edit the yml and env file and start again from #5.