Pip error on docker image 8.0.0.b32245

fcordova · November 8, 2019, 10:30pm

until v7 I could install packages on docker image but now it’s failing due permission errors…

ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/site-packages/elasticsearch-7.0.5.dist-info'
Consider using the `--user` option or check the permissions.

I guess using --user would not work when running server, would it?
how should I proceed to install new packages (in this case, elasticsearch) ?

levko · November 11, 2019, 2:57pm

Hi @fcordova! Can you please provide a full command(s) you run to build an image? Thanks!

fcordova · November 11, 2019, 3:39pm

Hi @levko, thanks for you reply…

I’m actually trying to create another image using the following dockerfile:

FROM redash/redash:8.0.0.b32245

RUN pip install elasticsearch

As you can see, pretty straightforward…

arikfr · November 11, 2019, 8:00pm

During the image’s build process one of the last commands we run is to switch the user to redash:

github.com

getredash/redash/blob/v8.0.0/Dockerfile#L25




# We first copy only the requirements file, to avoid rebuilding on every file
# change.
COPY requirements.txt requirements_bundles.txt requirements_dev.txt requirements_all_ds.txt ./
RUN pip install -r requirements.txt -r requirements_dev.txt
RUN if [ "x$skip_ds_deps" = "x" ] ; then pip install -r requirements_all_ds.txt ; else echo "Skipping pip install -r requirements_all_ds.txt" ; fi


COPY . /app
COPY --from=frontend-builder /frontend/client/dist /app/client/dist
RUN chown -R redash /app
USER redash


ENTRYPOINT ["/app/bin/docker-entrypoint"]
CMD ["server"]

In your Dockerfile you will need to change the user again before running the pip install command.

Btw, what are you installing elasticsearch for?

fcordova · November 11, 2019, 10:05pm

Hi arikfr… it worked fine, thanks…

I install elasticsearch because I have some python scripts that load data from it before processing the data to return it to Redash’s model…

‘You could use elasticsearch data source’, you’d say, but data in elasticsearch is not flat so I have to ‘unwind’ some collections to get the data I need. I also add some statistics, etc…

In mongodb I can create a pipeline to work data properly without using a python script, but I didn’t find a way to do that in elasticsearch

levko · November 13, 2019, 10:21am

If you need elasticsearch - no need to install it in Redash container; use power of Docker - just start separate elasticsearch container in the same network with Redash and you’ll be able to access it. If you use docker compose - you can just add a new service to docker-compose.yml

arikfr · November 13, 2019, 10:55am

Can you share an example for those scripts?

There is work being done to improve the Elasticsearch connector, so it might be useful to see real use cases.

github.com/getredash/redash

New Elasticsearch query runner

opened 09:19AM - 27 Oct 19 UTC

arikfr

Backend Query Runner: Elasticsearch

Our current Elasticsearch support is in a very bad state, it has issues with agg…regated queries, nested objects, and new versions of Elasticsearch. Also the codebase is in a very poor state not allowing for easy fixes. We are going to re-implement the connector, addressing these issues and making room for future improvements. Implementation notes: * Unless there will be request for it we will only support Elasticsearch's API and not the Kibana flavor. * This will be a new query runner. The current ones will be marked as deprecated, but existing queries using them will keep using them. * We will have support for SQL queries on top of Elasticsearch. There seem to be two flavors of it: community version and the X-Pack version. We should support both. Worth considering having the different SQL connectors as separate types from the ones that use the API directly (but reuse code, of course). Previous issues: * Aggregated queries issues: #2789, #3575, #3043 * Incompatibility with newer versions: #1744 * Other: #1596, #1075, #2791 Previous attempts at addressing these issues: * Support for elasticsearch-sql plugin: #2582 * Another SQL support: #3393 * Schema fetching errors: #3692 * Aggregated results support: #3845 * Nested fields support: #3456 * Nested aggregations support: #3809

▼Categories

▼Tags

Pip error on docker image 8.0.0.b32245