Update to version 8.0.0.0

I see. So that upgrade path has been deprecated since we use Docker now. The easiest option here is to migrate from your non-Docker instance to a Docker instance of the same version (V7 -> V7) and then upgrade your Docker instance to V8 using the steps shown in our “How To Upgrade” doc.

Can you direct me to that please

I updated my response with a link.

got it, thanks!!

but I don’t see the notes for the to migrate from your non-Docker instance to a Docker instance of the same version (V7 -> V7)

There aren’t official docs for this since the number of possible migration scenarios is near infinity. But it comes up on the forum a lot.

Have a look over this thread. Migration is pretty straightforward:

  1. Create a new instance of Redash V7 using docker.
  2. Import your database export from the non-Docker version

You’ll surely need to make other adjustments. But those are the basics.

2 Likes

Starting from the Redash V8 release we’re no longer building the tarball version of a release – only Docker images. This of course doesn’t prevent you from deploying Redash anyway you want, but the bin/upgrade script won’t work with the new releases.

Just noticed this on the web site and I’m now I am a bit stressed. So how do we upgrade without moving to docker. Step by step please.

Hi @bbrowski,

Can you share a bit info about your current deployment:

  1. What version are you running?
  2. Do you make backups of your database?
  3. What is the OS version?
  4. Is the database/redis run on the same machine?
1 Like

Hi @arikfr

  1. Currently running version 7.0.0.b17535
  2. I have backups of the redash db and the env file
  3. Ubuntu 18.04.3 LTS
  4. We have MySQL 5.7 database with most of the data that we access from redash queries running on the same machine. (not sure if that is what you are asking or if you are refering to some redash db - but in any case everything is on one machine and is working well)

Thanks

Hello, I have the same concerns.

Have been running redash in AWS for ages and upgraded to 7, now getting to 8 is a bit of a learning curve. Got 8 running in docker and in EC2 instance (requires 2Gb RAM, hence t2.small that is more expensive). Probably can get also 7 working in docker and then replace DB and upgrade, but

  1. how to replace DB within docker image?
  2. how to split docker image building and build ECS cluster with necessary networking?

There I need help. Is somewhere some examples or documentation that I could use?
Building images in EC2 instance and deploying them to ECS and figuring out networking? Is that the strategy to go for?

Also has someone been able to get redash in docker running on mac? As it would be significantly easier to do development and building on mac and then deploy to AWS ECS?

A lot of questions but I guess you get the point?

1 Like

There are two options for you to move to a Docker based deployment:

Move everything to Docker

The new AMIs have everything running in Docker. To move to such an instance:

  1. Create a backup of your Postgres database.
  2. Start a new Docker based AMI.
  3. Update the Docker image to use the same version as you were using previously (v7 in your case).
  4. Import the Postgres database backup you created in step 1.
  5. Check that everything is working properly.
  6. Upgrade to V8.

Move only Redash to Docker

The other option is to move only the Redash services to Docker, but keep the same Postgres/Redis you were using before on the host.

  1. Check how the setup script works to manually setup Docker and Docker-Compose.
  2. Use a similar docker-compose.yml, but remove the Postgres, Redis and Nginx services from it. Also you will need to setup it up in a way that allows the Docker containers to access ports from the host (to be able to communicate with Postgres/Redis).
  3. Update version to be the same version as you were using previously.
  4. Stop old services and start Docker Compose.
  5. Make sure that everything is working properly
  6. Upgrade to v8.

Obviously I skipped lots of details here, but wanted to provide the layout in case someone wants to take a stab at it.

Are you making changes to the default build that you need to build custom images?

I got docker based v8 working on EC2 but the goal is to move it all to ECS and drop EC2 instance. Not sure yet if that is more cheaper or not?

I was planning to build everything on docker and then deploy images to ECS and setup cluster and networking in a way where they can be talking to each other.

That would need docker compose rework I guess.
Added bonus would be to setup DB in RDS and have redash talking to it.

Sorry learning by doing here :slight_smile:

Running Redash’s services on ECS, having it proxied via ELB and the database on RDS is definitely a robust setup. It won’t be cheaper though :slight_smile:

We provide already built images of Redash, so this is why I asked why you need to build images?

1 Like

Roger, then I focus on getting v7 on docker to work on EC2 t2.small instance (was sofar running on micro with 1Gb and 1Gb swap) and update the DB and upgrade to v8. 2Gb RAM is enough I guess?

Well, it LOOKS like it’s upgraded - but it doesn’t. Worked OK during last upgrade. Using official Redash Docker image on AWS.

Did a reboot and everything, but my dashboard still says 7.0.0+b18042

Very often fails to restart after update and needs a reboot.

ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose stop server scheduler scheduled_worker adhoc_worker
Stopping redash_scheduled_worker_1 ... done
Stopping redash_scheduler_1        ... done
Stopping redash_adhoc_worker_1     ... done
Stopping redash_server_1           ... done
ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose run --rm server manage db upgrade
Starting redash_redis_1    ... done
Starting redash_postgres_1 ... done
[2019-11-13 17:50:45,983][PID:1][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-11-13 17:50:46,007][PID:1][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2019-11-13 17:50:47,898][PID:1][INFO][root] Latest version: 8.0.0 (newer: True)
[2019-11-13 17:50:48,512][PID:1][INFO][alembic.runtime.migration] Context impl PostgresqlImpl.
[2019-11-13 17:50:48,513][PID:1][INFO][alembic.runtime.migration] Will assume transactional DDL.
ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose up -d
redash_redis_1 is up-to-date
redash_postgres_1 is up-to-date
Starting redash_scheduled_worker_1 ... done
Starting redash_adhoc_worker_1     ... done
Starting redash_server_1           ... done
Starting redash_scheduler_1        ... done
redash_nginx_1 is up-to-date
ubuntu@ip-xx-xx-xx:/opt/redash$ sudo service supervisor restart all

docker-compose.yml

ersion: '2'
x-redash-service: &redash-service
  image: redash/redash:latest
  depends_on:
    - postgres
    - redis
  env_file: /opt/redash/env
  restart: always
services:
  server:
    <<: *redash-service
    command: server
    ports:
      - "5000:5000"
    environment:
      REDASH_WEB_WORKERS: 4
  scheduler:
    <<: *redash-service
    command: scheduler
    environment:
      QUEUES: "celery"
      WORKERS_COUNT: 1
  scheduled_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "scheduled_queries"
      WORKERS_COUNT: 1
  adhoc_worker:
    <<: *redash-service
    command: worker
    environment:
      QUEUES: "queries"
      WORKERS_COUNT: 2
  redis:
    image: redis:3.0-alpine
    restart: always
  postgres:
    image: postgres:9.5.6-alpine
    env_file: /opt/redash/env
    volumes:
      - /opt/redash/postgres-data:/var/lib/postgresql/data
    restart: always
  nginx:
    image: redash/nginx:latest
    ports:
      - "80:80"
    depends_on:
      - server
    links:
      - server:redash
    restart: always

Can you provide access to AMI with redash:7.0.0.b18042? I checked from AWS and could find only 5 and 8

There wasn’t one. By images I was referring to Docker images.

Roger, then I take AMI with 5 and upgrade to 7 via 6, replace DB and upgrade to 8?

Or build 7 from scratch on clean machine

Is there a third option? Could I build my own tarball? And if so how would I do that… I assume you used to do that every time from ver1 to 7.

Ok so Ive been testing out how to do this and I cant seem to get it to work. I tried for somthing close to the first option by trying to create a local vm to test it all out. I wont be bulding on a It’s a new ubuntu 18 instalation with nothing else on it.

I created the vm and copied the setup.sh file to the server. I edited the setup.sh to commented out
“sudo docker-compose run --rm server create_db
sudo docker-compose up -d”
I then ran the setup.sh and it all seemd to work. I edited the docker-compose.yml and changed redash:8.0.0.b32245 to redash:7.0.0.b18042.
I ran the docker-compose commands above to start the v7 instance.
Next I stoped all the docker containers and only restarted the redash_postgres_1 container.

So now I can import my old db, to do this I did the following:
sudo docker exec -i redash_postgres_1 dropdb postgres -U postgres
sudo docker exec -i redash_postgres_1 createdb postgres --owner=postgres -U postgres
sudo docker exec -i redash_postgres_1 psql -d postgres -U postgres < ~/backup.sql

I stoped the redash_postgres_1 container and restarted them all: sudo docker-compose up -d

I should now have a v7 docker running with my data from my non docker system. I can now acccess the server in the browser and my dashboard are there (but not working) but none of my queries will open. The queries are listed but when I click on them I get " It seems like we encountered an error. Try refreshing this page or contact your administrator."

My datasources are also missing so I tried to add them again - no joy. When I try and add a datasource (MySQL) I get a mesage telling me that a datasource with that name already exists. No matter what name I use I get the same error.

What am I doing wrong?

I cant upgrade to 8 till I get the docker working on 7 with my non docker data. I now understand a lot mor about dockers (but i still hate them).