Import custom python package/module for the python data source on dockerized redash server

Support Self Hosted Redash Support
1

Summary

I’m trying to make a custom python package usable inside a Redash query that use the python data source. The Redash server is self-hosted, running in a docker container (Debian GNU/Linux 10). The whole instance is managed by docker-compose. The Redash version is 9.0.0-beta (dev). For now, we are not able to update to a newer version of Redash.

What I tried

I created a test code python file, initially with only one method, let’s say test_method(). I copied the file onto the docker container to “/home/username/custom-python-for-redash/test_code.py”.
Then I restarted the Redash instance and added this very same absolute path to the AdditionalModulesPaths option of the python data source. When I try to use the code in a query, e.g. by typing

from test_code import test_method
print(test_method())

I get

Error running query: <class 'Exception'> 'test_code' is not configured as a supported import module

I also tried to make my python module a package to rule out the possibility that only proper packages are supported for import in Redash. For that, I moved it into a subdirectory (say: test_package) and created a __init__.py file. Then I copied said subdirectory to the container, like above, this time to “/home/username/custom-python-for-redash/test_package”.
I restarted the Redash instance again and added this absolute path to the AdditionalModulesPaths option. However, this yields the same error as written above.

In my despair I also tried (semi-random) variations for the absolute path, e.g. including or omitting the .py extension, using the path to the __init__.py file, using the path to the package directory instead of the actual file in it, etc. but with no results.

2

I also tried to install my local package on the Redash server docker container using pip. The installation succeeds and test_package is now listed when I execute pip list. However, with the data source configured as described in the question above, the error when trying to use the module in a query through the frontend still occurs.

The Redash Python data source documentation does not really help as well, since the information there is very limited and only describes what I already tried in the beginning.

Any help is appreciated.

3

Solution

I fixed the problem myself. The source folder containing the module in question was initially copied by a user other than the one that (automatically) instantiates the redash server container. The redash instance therefore had insufficient rights on the folder and could not read it, which is why modules in it weren’t found.

It is worth noting that in addition to writing the absolute path to the module into the AdditionalModulesPaths field, the name of the module inside that path has to be written into the field labelled “Modules to import prior to running the script” as well. After that, I could finally use import [...] to import and use my module.