I see. So that upgrade path has been deprecated since we use Docker now. The easiest option here is to migrate from your non-Docker instance to a Docker instance of the same version (V7 -> V7) and then upgrade your Docker instance to V8 using the steps shown in our “How To Upgrade” doc.
Can you direct me to that please
I updated my response with a link.
got it, thanks!!
but I don’t see the notes for the to migrate from your non-Docker instance to a Docker instance of the same version (V7 -> V7)
There aren’t official docs for this since the number of possible migration scenarios is near infinity. But it comes up on the forum a lot.
Have a look over this thread. Migration is pretty straightforward:
- Create a new instance of Redash V7 using docker.
- Import your database export from the non-Docker version
You’ll surely need to make other adjustments. But those are the basics.
Starting from the Redash V8 release we’re no longer building the tarball version of a release – only Docker images. This of course doesn’t prevent you from deploying Redash anyway you want, but the
bin/upgrade script won’t work with the new releases.
Just noticed this on the web site and I’m now I am a bit stressed. So how do we upgrade without moving to docker. Step by step please.
Can you share a bit info about your current deployment:
- What version are you running?
- Do you make backups of your database?
- What is the OS version?
- Is the database/redis run on the same machine?
- Currently running version 7.0.0.b17535
- I have backups of the redash db and the env file
- Ubuntu 18.04.3 LTS
- We have MySQL 5.7 database with most of the data that we access from redash queries running on the same machine. (not sure if that is what you are asking or if you are refering to some redash db - but in any case everything is on one machine and is working well)
Hello, I have the same concerns.
Have been running redash in AWS for ages and upgraded to 7, now getting to 8 is a bit of a learning curve. Got 8 running in docker and in EC2 instance (requires 2Gb RAM, hence t2.small that is more expensive). Probably can get also 7 working in docker and then replace DB and upgrade, but
- how to replace DB within docker image?
- how to split docker image building and build ECS cluster with necessary networking?
There I need help. Is somewhere some examples or documentation that I could use?
Building images in EC2 instance and deploying them to ECS and figuring out networking? Is that the strategy to go for?
Also has someone been able to get redash in docker running on mac? As it would be significantly easier to do development and building on mac and then deploy to AWS ECS?
A lot of questions but I guess you get the point?
There are two options for you to move to a Docker based deployment:
Move everything to Docker
The new AMIs have everything running in Docker. To move to such an instance:
- Create a backup of your Postgres database.
- Start a new Docker based AMI.
- Update the Docker image to use the same version as you were using previously (v7 in your case).
- Import the Postgres database backup you created in step 1.
- Check that everything is working properly.
- Upgrade to V8.
Move only Redash to Docker
The other option is to move only the Redash services to Docker, but keep the same Postgres/Redis you were using before on the host.
- Check how the setup script works to manually setup Docker and Docker-Compose.
- Use a similar
docker-compose.yml, but remove the Postgres, Redis and Nginx services from it. Also you will need to setup it up in a way that allows the Docker containers to access ports from the host (to be able to communicate with Postgres/Redis).
- Update version to be the same version as you were using previously.
- Stop old services and start Docker Compose.
- Make sure that everything is working properly
- Upgrade to v8.
Obviously I skipped lots of details here, but wanted to provide the layout in case someone wants to take a stab at it.
Are you making changes to the default build that you need to build custom images?
I got docker based v8 working on EC2 but the goal is to move it all to ECS and drop EC2 instance. Not sure yet if that is more cheaper or not?
I was planning to build everything on docker and then deploy images to ECS and setup cluster and networking in a way where they can be talking to each other.
That would need docker compose rework I guess.
Added bonus would be to setup DB in RDS and have redash talking to it.
Sorry learning by doing here
Running Redash’s services on ECS, having it proxied via ELB and the database on RDS is definitely a robust setup. It won’t be cheaper though
We provide already built images of Redash, so this is why I asked why you need to build images?
Roger, then I focus on getting v7 on docker to work on EC2 t2.small instance (was sofar running on micro with 1Gb and 1Gb swap) and update the DB and upgrade to v8. 2Gb RAM is enough I guess?
Well, it LOOKS like it’s upgraded - but it doesn’t. Worked OK during last upgrade. Using official Redash Docker image on AWS.
Did a reboot and everything, but my dashboard still says 7.0.0+b18042
Very often fails to restart after update and needs a reboot.
ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose stop server scheduler scheduled_worker adhoc_worker Stopping redash_scheduled_worker_1 ... done Stopping redash_scheduler_1 ... done Stopping redash_adhoc_worker_1 ... done Stopping redash_server_1 ... done ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose run --rm server manage db upgrade Starting redash_redis_1 ... done Starting redash_postgres_1 ... done [2019-11-13 17:50:45,983][PID:1][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt [2019-11-13 17:50:46,007][PID:1][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt [2019-11-13 17:50:47,898][PID:1][INFO][root] Latest version: 8.0.0 (newer: True) [2019-11-13 17:50:48,512][PID:1][INFO][alembic.runtime.migration] Context impl PostgresqlImpl. [2019-11-13 17:50:48,513][PID:1][INFO][alembic.runtime.migration] Will assume transactional DDL. ubuntu@ip-xx-xx-xx:/opt/redash$ sudo docker-compose up -d redash_redis_1 is up-to-date redash_postgres_1 is up-to-date Starting redash_scheduled_worker_1 ... done Starting redash_adhoc_worker_1 ... done Starting redash_server_1 ... done Starting redash_scheduler_1 ... done redash_nginx_1 is up-to-date ubuntu@ip-xx-xx-xx:/opt/redash$ sudo service supervisor restart all
ersion: '2' x-redash-service: &redash-service image: redash/redash:latest depends_on: - postgres - redis env_file: /opt/redash/env restart: always services: server: <<: *redash-service command: server ports: - "5000:5000" environment: REDASH_WEB_WORKERS: 4 scheduler: <<: *redash-service command: scheduler environment: QUEUES: "celery" WORKERS_COUNT: 1 scheduled_worker: <<: *redash-service command: worker environment: QUEUES: "scheduled_queries" WORKERS_COUNT: 1 adhoc_worker: <<: *redash-service command: worker environment: QUEUES: "queries" WORKERS_COUNT: 2 redis: image: redis:3.0-alpine restart: always postgres: image: postgres:9.5.6-alpine env_file: /opt/redash/env volumes: - /opt/redash/postgres-data:/var/lib/postgresql/data restart: always nginx: image: redash/nginx:latest ports: - "80:80" depends_on: - server links: - server:redash restart: always
Can you provide access to AMI with redash:7.0.0.b18042? I checked from AWS and could find only 5 and 8
There wasn’t one. By images I was referring to Docker images.
Roger, then I take AMI with 5 and upgrade to 7 via 6, replace DB and upgrade to 8?
Or build 7 from scratch on clean machine
Is there a third option? Could I build my own tarball? And if so how would I do that… I assume you used to do that every time from ver1 to 7.
Ok so Ive been testing out how to do this and I cant seem to get it to work. I tried for somthing close to the first option by trying to create a local vm to test it all out. I wont be bulding on a It’s a new ubuntu 18 instalation with nothing else on it.
I created the vm and copied the setup.sh file to the server. I edited the setup.sh to commented out
“sudo docker-compose run --rm server create_db
sudo docker-compose up -d”
I then ran the setup.sh and it all seemd to work. I edited the docker-compose.yml and changed redash:8.0.0.b32245 to redash:7.0.0.b18042.
I ran the docker-compose commands above to start the v7 instance.
Next I stoped all the docker containers and only restarted the redash_postgres_1 container.
So now I can import my old db, to do this I did the following:
sudo docker exec -i redash_postgres_1 dropdb postgres -U postgres
sudo docker exec -i redash_postgres_1 createdb postgres --owner=postgres -U postgres
sudo docker exec -i redash_postgres_1 psql -d postgres -U postgres < ~/backup.sql
I stoped the redash_postgres_1 container and restarted them all: sudo docker-compose up -d
I should now have a v7 docker running with my data from my non docker system. I can now acccess the server in the browser and my dashboard are there (but not working) but none of my queries will open. The queries are listed but when I click on them I get " It seems like we encountered an error. Try refreshing this page or contact your administrator."
My datasources are also missing so I tried to add them again - no joy. When I try and add a datasource (MySQL) I get a mesage telling me that a datasource with that name already exists. No matter what name I use I get the same error.
What am I doing wrong?
I cant upgrade to 8 till I get the docker working on 7 with my non docker data. I now understand a lot mor about dockers (but i still hate them).