1

Issue Summary:

I had to restart our EC2 instance as it was running out of space, All I did at first was to increase the volume size.
After The instance had restarted I noted that the public Ip had changed, and I could no longer access the UI.

I did check the status of all the containers, and it seem like Postgres did not properly start I then tried to force rebuild all the containers and I could see this error:

redash_postgres_1 exited with code 1

$ docker ps:

08700e4a5325   postgres:9.6-alpine               "docker-entrypoint.s…"   3 hours ago   Restarting (1) 14 seconds ago                                                redash_postgres_1`

$ sudo docker-compose up --force-recreate --build -d

redash_postgres_1 docker-entrypoint.sh postgres Restarting

postgres_1          | PostgreSQL Database directory appears to contain a database; Skipping initialization
postgres_1          |
postgres_1          | LOG:  database system was shut down at 2021-11-23 12:28:52 UTC
postgres_1          | LOG:  invalid record length at 6/17E95520: wanted 24, got 0
postgres_1          | LOG:  invalid primary checkpoint record
postgres_1          | LOG:  invalid resource manager ID in secondary checkpoint record
postgres_1          | PANIC:  could not locate a valid checkpoint record
postgres_1          | LOG:  startup process (PID 22) was terminated by signal 6: Aborted
postgres_1          | LOG:  aborting startup due to startup process failure
postgres_1          | LOG:  database system is shut down`

When trying to access the database I get this:
psql: could not translate host name “postgres” to address: Try again

Thank you for your assistance

Technical details:

  • Redash Version: V9
  • Browser/OS: Chrome
  • How did you install Redash: Docker
2

I edited your post to make your terminal outputs more reasonable.

Pro tip: use triple backticks above and below a block to do this yourself.

Rebuilding all the containers was not necessary here.

Can you share your docker-compose file? Why are none of your other containers running?

3

Thanks for the pro tip :slight_smile:
Bellow the Docker-compose.yml file

/opt/redash$ sudo nano docker-compose.yml

version: '2'
x-redash-service: &redash-service
  image: redash/redash:9.0.0-beta.b42121
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always
services:
  worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "periodic emails default"
      WORKERS_COUNT: 1
  server:
    <<: *redash-service
    command: server
    ports:
      - "5000:5000"
    environment:
      REDASH_WEB_WORKERS: 4
  scheduler:
    <<: *redash-service
    command: scheduler
  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries,schemas"
      WORKERS_COUNT: 1
  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2
  redis:
    image: redis:5.0-alpine
    restart: always
  postgres:
    image: postgres:9.6-alpine
    environment:
      - "POSTGRES_HOST_AUTH_METHOD=trust"
    ports:
      - "5432:5432"
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    restart: always
  nginx:
    image: redash/nginx:latest
    ports:
      - "80:80"
    depends_on:
      - server
    links:
      - server:redash
    restart: always

I believe that aside Postgres all the other containers are up

/opt/redash$ docker ps
CONTAINER ID   IMAGE                             COMMAND                  CREATED        STATUS                          PORTS                                        NAMES
ecfbf2a07110   redash/redash:9.0.0-beta.b42121   "/app/bin/docker-ent…"   12 hours ago   Up 9 hours                      5000/tcp                                     redash_adhoc_worker_1
46bfd1697384   redash/redash:9.0.0-beta.b42121   "/app/bin/docker-ent…"   12 hours ago   Up 9 hours                      5000/tcp                                     redash_scheduled_worker_1
126c437edf4d   redash/redash:9.0.0-beta.b42121   "/app/bin/docker-ent…"   12 hours ago   Up 9 hours                      5000/tcp                                     redash_worker_1
352ef41688e1   redash/nginx:latest               "nginx -g 'daemon of…"   12 hours ago   Up 9 hours                      0.0.0.0:80->80/tcp, :::80->80/tcp, 443/tcp   redash_nginx_1
01f4b9f9b135   redash/redash:9.0.0-beta.b42121   "/app/bin/docker-ent…"   12 hours ago   Up 9 hours                      0.0.0.0:5000->5000/tcp, :::5000->5000/tcp    redash_server_1
65db0fe55db3   redash/redash:9.0.0-beta.b42121   "/app/bin/docker-ent…"   12 hours ago   Up 9 hours                      5000/tcp                                     redash_scheduler_1
70e01f210483   redis:5.0-alpine                  "docker-entrypoint.s…"   12 hours ago   Up 9 hours                      6379/tcp                                     redash_redis_1
08700e4a5325   postgres:9.6-alpine               "docker-entrypoint.s…"   12 hours ago   Restarting (1) 42 seconds ago                                                redash_postgres_1