1

I can successfully create a Python Data Source and use it, but now I’m trying to use various pip libraries (like awswrangler), which aren’t installed in the docker image.

My question is how do I go about making these packages available to the Python Data Sources that I create within Redash?

I guess I could exec into the redash worker containers and pip install them manually, but I’d need to redo this work every time the container was stopped.

I’m thinking another way of doing this is to edit the docker-compose.yml file, add a volume my host machine’s site-packages for Python, and then add that volume as an “Additional Module Path” to the data source. I’m also assuming that would require create a virtual environment to ensure the packages are installed using the same version of python that’s being used in the Redash docker image? But, I’m not sure which container to do this in, and I can’t seem to figure out the version of Python being used. The Dockerfile sure looks like it’s 3.7, but executions of docker-compose exec <container> python --version show 2.7 and python3 isn’t in the PATH of any of the containers.

How do others solve this issue? Thanks!

Edit: the images in question were built using the AWS AMI. If that’s old and I should be using an updated image (e.g. docker-compose pull) that’d be good to know!

2

Any of those solutions would work. I agree with all the pitfalls you specified.

Generally, we need to publish better docs around this.

That said, modifying the source the AMI deployments isn’t easy. But it’s not really supposed to be. The AMIs are meant to address the most common use-case of Redash: click a button to deploy, connect a sql database, get to work.

For a more complex use-case like modifying the source code or tweaking the Dockerfile to install extra packages to the worker Python site-packages I’d go the more typical setup route where you build the docker image from source.

With this approach, you can modify Redash’s Dockerfile to install your desired dependencies, build an image with the modified Dockerfile, and use that in your docker-compose instead of the default dockerhub tag (you can see an example of this in the docker-compose.yml file at the root of our repo).