First let me say that I love Redash. It’s brilliant.
I was a little disappointed that the new version 5 seems to no longer have an easy way to install redash on your own machine without using a docker (https://redash.io/help/open-source/setup). Im sure there are other who dont want to use dockers for some or other reason. The new Setup Script creates a docker installation.
Lucky for me I still had a copy of the old bootstrap.sh file used in v4. I used this and then did a upgrade from v4 to v5.
Could you please give us an updated script to install redash 5 without using dockers.
Alternatively give us some clearer instructions than the ones in https://redash.io/help/open-source/dev-guide/setup. When I read it I had too many questions (think step by step installation guide for noobs). Again I was glad for the old bootstrap.sh and it will probably be easier to bring an updated version of this back?
We deprecated the old setup in favor of the new one for the following reasons:
It’s more robust: now whenever we ship an upgrade it comes with all the needed dependencies. So no more cases where you upgrade your instance and some query runners no longer work. Also, it will allow us to do bigger changes (like switch to Python 3 from Legacy Python) with ease, as it’s transparent to the user.
We’re a small team and maintaining two deployment options is not something we can do.
Having said that if there is a good use case for the non Docker deployment we can see how we can make it a community supported effort or improve the documentation. I just want to understand the use case better first.
Dont dockers use more resources and slow down performance? It’s ok for testing out on a vm but why would I want a docker on a production machine?
When I was looking for an open source dashboard solution I tested out redash and metabase. Metabase came in a docker and it was a pain (it also had other limitations that redash did not have). So my docker experience was painful. It felt like I needed to learn a whole new way of doing things.
I get the small team thing (you are doing a great job) so I understand why you wound not be able to commit the resource to two deployment options. I’ll look at the https://redash.io/help/open-source/dev-guide/setup and see if I can do it on a fresh vm - and make notes as I go.
Hi there Arik. I too opted-out of docker installs. I’m using lxd to manage my linux apps. These work as separated linux boxes and not as containers like docker. This way we can work with usual tool set and not mess with endless docker* tools and configs.
It is a bit worrying that you now force users to go docker or cloud to get redash. That is a huge constraint and had I known it before I would not have selected redash while evaluating existing options.
In the scenario where you uprade dependencies I would create a new clean lxd ubunut machine install redash vX and upgrade the instance database… Can’t see the issue there.
On non “native” operating systems like MacOS or Windows Docker is used via a VM, and hence has some overhead. But on a Linux machine for an application like Redash the overhead of using Docker is minimal (if existing). I do know that running databases might introduce some challenges with Docker and certain scale, because of the filesystem use. But we recommend splitting the database to a separate instance anyway for serious use
I would really not be worried about Docker’s overhead for Redash’s own processes (that is everything except Postgres and Redis). The benefits it provides in terms of ease of deployment definitely worth it.
You can also use the old bootstrap script as reference – I doubt a lot changes since then.
But I encourage you to give the Docker based instances a try to see if the experience changed since the previous time you gave Docker a chance.
We don’t force anyone to do anything Redash is still open source and I’m sure that an experienced operator will figure out how to deploy it whatever way they need it.
It just that the default deployment we support is using Docker. This makes it easier both for us and the end user.
I will check about lxd, but Docker has the benefit of being more widely adopted and the image more portable – can be used by those who use Docker Compose, Kubernetes, ECS, etc.
I edited the old bootstrap and tried it out on a new clean Ubuntu 18.04.1 LTS VM. It all seems to install correctly except for when it is doing the “pip install -r $REDASH_BASE_PATH/current/requirements_all_ds.txt.”
At this point boto3 1.9.26, influxdb 2.7.1 and memsql 2.16.0 dont seem to install because there are new versions of the files they are dependent on (eg six). The rest of the script works. I had an idea to edit the requirements_all_ds.txt file and add a > sign to those line items but didnt want to break it more.
the rest of the instalation seems to work. But when I try to access the site after the instalation compleates it does not look right:
If I enter details in and click setup I get a “Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.”
This is how i edited the bootstrap.sh file:
#!/bin/bash
#
# This script setups Redash along with supervisor, nginx, PostgreSQL and Redis. It was written to be used on
# Ubuntu 18.04. Technically it can work with other Ubuntu versions, but you might get non compatible versions
# of PostgreSQL, Redis and maybe some other dependencies.
#
# This script is not idempotent and if it stops in the middle, you can't just run it again. You should either
# understand what parts of it to exclude or just start over on a new VM (assuming you're using a VM).
set -eu
REDASH_BASE_PATH=/opt/redash
REDASH_BRANCH="${REDASH_BRANCH:-master}" # Default branch/version to master if not specified in REDASH_BRANCH env var
REDASH_VERSION=${REDASH_VERSION-5.0.1} # Install latest version if not specified in REDASH_VERSION env var
LATEST_URL="https://github.com/getredash/redash.git"
VERSION_DIR="$REDASH_BASE_PATH/redash.${REDASH_VERSION}"
REDASH_TARBALL=/tmp/redash.tar.gz
FILES_BASE_URL=https://raw.githubusercontent.com/getredash/redash/release/4.0.x/setup/ubuntu/files
cd /tmp/
verify_root() {
# Verify running as root:
if [ "$(id -u)" != "0" ]; then
if [ $# -ne 0 ]; then
echo "Failed running with sudo. Exiting." 1>&2
exit 1
fi
echo "This script must be run as root. Trying to run with sudo."
sudo bash "$0" --with-sudo
exit 0
fi
}
create_redash_user() {
adduser --system --no-create-home --disabled-login --gecos "" redash
}
install_system_packages() {
apt-get -y install software-properties-common
apt-add-repository universe
apt-get -y update
# Base packages
apt install -y python-pip python-dev nginx curl build-essential pwgen
# Data sources dependencies:
apt install -y libffi-dev libssl-dev libmysqlclient-dev libpq-dev freetds-dev libsasl2-dev
# SAML dependency
apt install -y xmlsec1
# Storage servers
apt install -y postgresql redis-server
apt install -y supervisor
}
create_directories() {
mkdir -p $REDASH_BASE_PATH
chown redash $REDASH_BASE_PATH
# Default config file
if [ ! -f "$REDASH_BASE_PATH/.env" ]; then
sudo -u redash wget "$FILES_BASE_URL/env" -O $REDASH_BASE_PATH/.env
fi
COOKIE_SECRET=$(pwgen -1s 32)
echo "export REDASH_COOKIE_SECRET=$COOKIE_SECRET" >> $REDASH_BASE_PATH/.env
}
extract_redash_sources() {
# sudo -u redash wget "$LATEST_URL" -O "$REDASH_TARBALL"
# sudo -u redash mkdir "$VERSION_DIR"
sudo -u redash git clone "$LATEST_URL" "$VERSION_DIR"
# sudo -u redash tar -C "$VERSION_DIR" -xvf "$REDASH_TARBALL"
ln -nfs "$VERSION_DIR" $REDASH_BASE_PATH/current
ln -nfs $REDASH_BASE_PATH/.env $REDASH_BASE_PATH/current/.env
}
install_python_packages() {
sudo pip install --upgrade pip
# TODO: venv?
pip install setproctitle # setproctitle is used by Celery for "pretty" process titles
pip install -r $REDASH_BASE_PATH/current/requirements.txt
pip install -r $REDASH_BASE_PATH/current/requirements_all_ds.txt
}
create_database() {
# Create user and database
sudo -u postgres createuser redash --no-superuser --no-createdb --no-createrole
sudo -u postgres createdb redash --owner=redash
cd $REDASH_BASE_PATH/current
sudo -u redash bin/run ./manage.py database create_tables
}
setup_supervisor() {
wget -O /etc/supervisor/conf.d/redash.conf "$FILES_BASE_URL/supervisord.conf"
service supervisor restart
}
setup_nginx() {
rm /etc/nginx/sites-enabled/default
wget -O /etc/nginx/sites-available/redash "$FILES_BASE_URL/nginx_redash_site"
ln -nfs /etc/nginx/sites-available/redash /etc/nginx/sites-enabled/redash
service nginx restart
}
verify_root
install_system_packages
create_redash_user
create_directories
extract_redash_sources
install_python_packages
create_database
setup_supervisor
setup_nginx
You should turn it into a code block by wrapping with “```”.
Based on how it looks, it seems that the compiled static assets are missing.Did you use one of the release tarballs we provide or downloaded codebase from GitHub?
Ok I see - I was installing the codebase. Where are the tarballs - Ive been looking for half an hour and cant find the location. If I knew where they were then I could fix it.
I did try with the original file but for some reason it would not find the file - I must have typed something incorrectly - so I tried making a git clone. I now realise you cant just do that. Anyway It was a good learning experience.
The script below works (for others who want to install the latest redash version easily without using dockers) on Ubuntu 18.04.1 LTS.
#!/bin/bash
#
# This script setups Redash along with supervisor, nginx, PostgreSQL and Redis. It was written to be used on
# Ubuntu 18.04. Technically it can work with other Ubuntu versions, but you might get non compatible versions
# of PostgreSQL, Redis and maybe some other dependencies.
#
# This script is not idempotent and if it stops in the middle, you can't just run it again. You should either
# understand what parts of it to exclude or just start over on a new VM (assuming you're using a VM).
set -eu
REDASH_BASE_PATH=/opt/redash
REDASH_BRANCH="${REDASH_BRANCH:-master}" # Default branch/version to master if not specified in REDASH_BRANCH env var
REDASH_VERSION=${REDASH_VERSION-5.0.2.b5485} # Install latest version if not specified in REDASH_VERSION env var
LATEST_URL="https://s3.amazonaws.com/redash-releases/redash.${REDASH_VERSION}.tar.gz"
VERSION_DIR="$REDASH_BASE_PATH/redash.${REDASH_VERSION}"
REDASH_TARBALL=/tmp/redash.tar.gz
FILES_BASE_URL=https://raw.githubusercontent.com/getredash/redash/release/4.0.x/setup/ubuntu/files
cd /tmp/
verify_root() {
# Verify running as root:
if [ "$(id -u)" != "0" ]; then
if [ $# -ne 0 ]; then
echo "Failed running with sudo. Exiting." 1>&2
exit 1
fi
echo "This script must be run as root. Trying to run with sudo."
sudo bash "$0" --with-sudo
exit 0
fi
}
create_redash_user() {
adduser --system --no-create-home --disabled-login --gecos "" redash
}
install_system_packages() {
# these 2 lines are needed to get pip installed
apt-get -y install software-properties-common
apt-add-repository universe
apt-get -y update
# Base packages
apt install -y python-pip python-dev nginx curl build-essential pwgen
# Data sources dependencies:
apt install -y libffi-dev libssl-dev libmysqlclient-dev libpq-dev freetds-dev libsasl2-dev
# SAML dependency
apt install -y xmlsec1
# Storage servers
apt install -y postgresql redis-server
apt install -y supervisor
}
create_directories() {
mkdir -p $REDASH_BASE_PATH
chown redash $REDASH_BASE_PATH
# Default config file
if [ ! -f "$REDASH_BASE_PATH/.env" ]; then
sudo -u redash wget "$FILES_BASE_URL/env" -O $REDASH_BASE_PATH/.env
fi
COOKIE_SECRET=$(pwgen -1s 32)
echo "export REDASH_COOKIE_SECRET=$COOKIE_SECRET" >> $REDASH_BASE_PATH/.env
}
extract_redash_sources() {
sudo -u redash wget "$LATEST_URL" -O "$REDASH_TARBALL"
sudo -u redash mkdir "$VERSION_DIR"
sudo -u redash tar -C "$VERSION_DIR" -xvf "$REDASH_TARBALL"
ln -nfs "$VERSION_DIR" $REDASH_BASE_PATH/current
ln -nfs $REDASH_BASE_PATH/.env $REDASH_BASE_PATH/current/.env
}
install_python_packages() {
sudo pip install --upgrade pip
pip install setproctitle # setproctitle is used by Celery for "pretty" process titles
pip install -r $REDASH_BASE_PATH/current/requirements.txt
pip install -r $REDASH_BASE_PATH/current/requirements_all_ds.txt
}
create_database() {
# Create user and database
sudo -u postgres createuser redash --no-superuser --no-createdb --no-createrole
sudo -u postgres createdb redash --owner=redash
cd $REDASH_BASE_PATH/current
sudo -u redash bin/run ./manage.py database create_tables
}
setup_supervisor() {
wget -O /etc/supervisor/conf.d/redash.conf "$FILES_BASE_URL/supervisord.conf"
service supervisor restart
}
setup_nginx() {
rm /etc/nginx/sites-enabled/default
wget -O /etc/nginx/sites-available/redash "$FILES_BASE_URL/nginx_redash_site"
ln -nfs /etc/nginx/sites-available/redash /etc/nginx/sites-enabled/redash
service nginx restart
}
verify_root
install_system_packages
create_redash_user
create_directories
extract_redash_sources
install_python_packages
create_database
setup_supervisor
setup_nginx
Thanks for this post.
I am trying to install redash 5 by running this script need (as I also like the conventional approach better) 1: there is some problem in aouthlib.common library as bytes_type module is not getting imported.
File “/usr/local/lib/python2.7/dist-packages/flask_oauthlib/utils.py”, line 5, in
from oauthlib.common import to_unicode, bytes_type
ImportError: cannot import name bytes_type
Thanks for the valuable information and for the updated script (@bbrowski and @arikfr). As the other guys, I’d rather install Redash without Docker.
Here with a clean Ubuntu 18.04 server, I’ve faced the same “ImportError: cannot import name bytes_type” problem described above. On the other hand, I didn’t have the FILES_BASE_URL 404 error (at least I didn’t have it yet ).
As far as I can see, the 4.0.x release tag on Github project is still active and it has a setup/ubuntu/files folder:
Sorry if it’s just a guess (I’m not an expert with deployment) but it seems necessary to restrict the version of oauthlib used by Redash 5. I’ll do some tests here and report whether it works or not.
Thanks, @arikfr! I was just rebuilding my system from ground up but was thinking about trying with 0.9.4. I’ve came to the same conclusion that it was probably Flask-OAuthLib the one that was importing an incompatible version of oauthlib…
I’ve just completed the setup of a clean Ubuntu 18.04 server with Redash 5 with Flask-OAuthLib 0.9.5.
Unfortunately I’ve got some incompatibility messages almost at the end of the process:
boto3 1.9.80 has requirement botocore<1.13.0,>=1.12.80, but you’ll have botocore 1.10.2 which is incompatible.
pandas 0.23.4 has requirement python-dateutil>=2.5.0, but you’ll have python-dateutil 2.4.2 which is incompatible.
influxdb 2.7.1 has requirement six==1.9.0, but you’ll have six 1.10.0 which is incompatible.
memsql 2.16.0 has requirement python-dateutil==2.2, but you’ll have python-dateutil 2.4.2 which is incompatible.
Setup completed normally but I’m getting a “504 Gateway Time-out” error while trying to access Redash 5. I’ll check if it’s somewhat related to these problems listed above.
It finally worked! Now I have a 5.0.2 Redash VM here without docker. I had to perform a setup in two phases, being the first one with the top 5 main commands:
).