Exact steps needed for upgrade?

Issue Summary

It turns out the docker image based redash I’m using is now outdated (v5), whereas there’s a newer (v6) version available.

However, the “documentation” on how to upgrade seems to be more of a high level overview, missing the more detailed specifics needed by new Redash users. For example, the first line of the current doc:

Make sure to backup your data. You only need to backup your PostgreSQL metadata as the data in Redis is transient.

For someone new to Redash, that’s really unclear. Which tables contain the PostgreSQL metadata?

These are the tables in my currently running (v5) instance of the redash_postgres container:

 public | access_permissions        | table | postgres
 public | alembic_version           | table | postgres
 public | alert_subscriptions       | table | postgres
 public | alerts                    | table | postgres
 public | api_keys                  | table | postgres
 public | changes                   | table | postgres
 public | dashboards                | table | postgres
 public | data_source_groups        | table | postgres
 public | data_sources              | table | postgres
 public | events                    | table | postgres
 public | favorites                 | table | postgres
 public | groups                    | table | postgres
 public | notification_destinations | table | postgres
 public | organizations             | table | postgres
 public | queries                   | table | postgres
 public | query_results             | table | postgres
 public | query_snippets            | table | postgres
 public | users                     | table | postgres
 public | visualizations            | table | postgres
 public | widgets                   | table | postgres

After further mucking around and trying things out, the line in the doc should probably instead say:

Make sure to backup your data. You only need to backup PostgreSQL (the entire database), as data in Redis is transient.

That would be much clearer.

The fully worked commands to perform the backup and restore would probably be useful too.

For my local development setup, I retrieved the IP address of the PostgreSQL container using:

$ docker inspect --format '{{ .NetworkSettings.Networks.redash_default.IPAddress }}' redash_postgres_1

Which gave:

172.18.0.3

Backing up the Redash database then became:

$ pg_dump -h 172.18.0.3 -U postgres -c postgres > backup.sql

---

To restore the Redash database, it’s a matter of:

1. Shut down all the Redash containers other than PostgreSQL

    $ docker-compose stop
    Stopping redash_nginx_1            ... done
    Stopping redash_adhoc_worker_1     ... done
    Stopping redash_server_1           ... done
    Stopping redash_scheduler_1        ... done
    Stopping redash_scheduled_worker_1 ... done
    Stopping redash_postgres_1         ... done
    Stopping redash_redis_1            ... done
    $ docker start redash_postgres_1
    redash_postgres_1

2. Retrieve the new IP address of the PostgreSQL container, as it will have likely changed when restarted:

    $ docker inspect --format '{{ .NetworkSettings.Networks.redash_default.IPAddress }}' redash_postgres_1
    172.18.0.2

3. Restore the Redash database

    $ psql -h 172.18.0.2 -U postgres postgres < backup.sql
    ALTER TABLE
    ...

4. Start the remaining Redash containers:

    $ docker-compose start
    Starting redis            ... done
    Starting postgres         ... done
    Starting adhoc_worker     ... done
    Starting scheduled_worker ... done
    Starting scheduler        ... done
    Starting server           ... done
    Starting nginx            ... done

5. Verify the Redash containers are all running:

    $ docker-compose ps
              Name                         Command               State              Ports           
    ------------------------------------------------------------------------------------------------
    redash_adhoc_worker_1       /app/bin/docker-entrypoint ...   Up      5000/tcp                   
    redash_nginx_1              nginx -g daemon off;             Up      443/tcp, 0.0.0.0:80->80/tcp
    redash_postgres_1           docker-entrypoint.sh postgres    Up      5432/tcp                   
    redash_redis_1              docker-entrypoint.sh redis ...   Up      6379/tcp                   
    redash_scheduled_worker_1   /app/bin/docker-entrypoint ...   Up      5000/tcp                   
    redash_scheduler_1          /app/bin/docker-entrypoint ...   Up      5000/tcp                   
    redash_server_1             /app/bin/docker-entrypoint ...   Up      0.0.0.0:5000->5000/tcp

That’s probably detailed enough for new users, and suitable for cut-n-pasting into the upgrade doc.

Thank you, @justinclift for this detailed post! And apologies for the unclear reference about the PostgreSQL database. The reason we mention “metadata” there is to avoid confusion with users who have their own PostgreSQL database they query with Redash. But I guess we just created another confusion point

I hope this version is better:

Thanks @arikfr . Yep, that’s clearer.

As a data point, using pg_dump for backing up the database probably isn’t needed.

The Docker image based approach creates a PostgreSQL data directory under /opt/redash, so just shutting down the containers (docker-compose stop) then copying that directory structure to a new location (as the backup), should suffice.

Interesting! Yeah, I guess this should be good enough for upgrades. But for regular periodic backups, pg_dump is probably better as you can run it without stopping anything.

Good point. I’m still at the “setting things up in a dev environment” stage.

---

Got that working, but it’s a bit wonky. Not really much in the way of documentation (yet?) for an optimal Redash dev setup. eg is the expected approach to run the containers, bind the (local development git clone) to the redash /app directory or something, etc?

Anyway, I seem to have it working “enough” at the moment for what I need, so all good.

Have you seen this KB page: https://redash.io/help/open-source/dev-guide/docker ?

Thanks Arik. That looks like it would get the job done.

Hello @justinclift ,

Thank you for the backup & restore steps.
can you please clarify my doubt on below point,

1. In our Redash Linux VM, when I checked there are multiple Postgres Connection strings present, is it due to multiple data sources on redash instance ? And do we need to take backup of each before upgrading ?

Please assist on urgent basis.

Thanks,
Mandar