Unable to docker-compose build

i’m on the cutting edge latest a2217cc
cannot do docker-compose build:

Setting up g++ (4:6.3.0-4) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up libsasl2-modules-gssapi-mit:amd64 (2.1.27~101-g0780600+dfsg-3+deb9u1) ...
Setting up openssh-client (1:7.4p1-10+deb9u7) ...
Setting up libsybdb5:amd64 (0.91-6.1+b4) ...
Setting up libct4:amd64 (0.91-6.1+b4) ...
Setting up libpq-dev (9.6.17-0+deb9u1) ...
Setting up build-essential (12.3) ...
Setting up libcurl3:amd64 (7.52.1-5+deb9u10) ...
Setting up libcurl3-gnutls:amd64 (7.52.1-5+deb9u10) ...
Setting up freetds-dev (0.91-6.1+b4) ...
Setting up git (1:2.11.0-3+deb9u7) ...
Setting up curl (7.52.1-5+deb9u10) ...
Setting up git-core (1:2.11.0-3+deb9u7) ...
Setting up odbcinst1debian2:amd64 (2.3.4-1) ...
Setting up unixodbc-dev (2.3.4-1) ...
Setting up odbcinst (2.3.4-1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for sgml-base (1.29) ...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0Warning: apt-key output should not be parsed (stdout is not a terminal)
100   983  100   983    0     0    174      0  0:00:05  0:00:05 --:--:--   238
OK
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    74  100    74    0     0     13      0  0:00:05  0:00:05 --:--:--    23
Ign:1 http://deb.debian.org/debian stretch InRelease
Hit:2 http://deb.debian.org/debian stretch-updates InRelease
Hit:3 http://deb.debian.org/debian stretch Release
Hit:5 http://security.debian.org/debian-security stretch/updates InRelease
Reading package lists...
E: The method driver /usr/lib/apt/methods/https could not be found.
E: Failed to fetch https://packages.microsoft.com/debian/10/prod/dists/buster/InRelease
E: Some index files failed to download. They have been ignored, or old ones used instead.
ERROR: Service 'server' failed to build: The command '/bin/sh -c apt-get update &&   apt-get install -y     curl     gnupg     build-essential     pwgen     libffi-dev     sudo     git-core     wget     libpq-dev     g++ unixodbc-dev     xmlsec1     libssl-dev     default-libmysqlclient-dev     freetds-dev     libsasl2-dev     unzip     libsasl2-modules-gssapi-mit &&   curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - &&   curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list &&   apt-get update &&   ACCEPT_EULA=Y apt-get install -y msodbcsql17 &&   apt-get clean &&   rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Makefile:4: recipe for target 'compose_build' failed
make: *** [compose_build] Error 1

i added apt-transport-https ca-certificates to the apt-get install -y list and was able to proceed to step 23/30:

Step 23/30 : RUN if [ "x$skip_dev_deps" = "x" ] ; then pip install -r requirements.txt -r requirements_dev.txt; else pip install -r requirements.txt; fi
 ---> Running in 4d81d3ee18d0
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 259, in run
    with self._build_session(options) as session:
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 76, in _build_session
    if options.cache_dir else None
  File "/usr/local/lib/python3.7/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not int
ERROR: Service 'server' failed to build: The command '/bin/sh -c if [ "x$skip_dev_deps" = "x" ] ; then pip install -r requirements.txt -r requirements_dev.txt; else pip install -r requirements.txt; fi' returned a non-zero code: 2
Makefile:4: recipe for target 'compose_build' failed
make: *** [compose_build] Error 1

Please don’t open issues against our open source repository for stuff like this. You’ve written about it already on the forum, which is the right spot.

My guess is that you have a firewall blocking installation of some dependencies. Have you tried opening up the firewall for the duration of this build?

no firewalls… i started here in the forum but realized it was a bug, wasn’t it? especially the second part. didn’t you read the solution part of my github post?

Solution:

  1. Need apt-transport-https ca-certificates in the apt-get install list
  2. At this line here, in order to disable pip cache, use ENV PIP_NO_CACHE_DIR=off not ENV PIP_NO_CACHE_DIR=1 . See pypa/pip#2897

I realize the line between “support question” and “bug report” is a bit fuzzy here. I did read your solution and I’m glad it worked for you. But as far as I can tell this isn’t a bug in our setup. I tried and can’t reproduce your error. The tip of master builds successfully on my local box (MacOS Catalina) and on an Ubuntu VM. And speaking generally, our team develops off the master branch so if there truly were a critical build error we’d know about it already.

I don’t have firm stats. But it feels like 99% of build errors are not an issue with our Dockerfile or configuration. I don’t have reason to believe this one is different. Which is why I closed your issue on the repo and if necessary we can continue troubleshooting here.

If you feel I’m off-base we can reopen your issue. But since we can’t reproduce the error you see there’s not much we can do about it :man_shrugging:

ok, fair enough… but still feels weird. i’ve been following redash since 2015. this is the 1st time i have to edit Dockerfile to make it work :thinking:

I just finished a clean build with no changes to the Dockerfile, but:

  1. Yesterday I did have a similar issue as you had with installing Debian packages. I couldn’t attend to it and today it’s gone. In the past I had the same issues when our ISP was doing funny things with proxying. Can you try again without the packages you added?

  2. We didn’t have any issues with this option before :expressionless: It seems like a pip version issue. Can you check what version of pip your image has?

When trying to build again, can you add the flag --pull=true to the build command? It might be that you have an older base image.

yeah the 1st one is gone now, still don’t know the exact reason though
the 2nd one is gone too probably because of the update pushed 11 hours ago in docker-library/ python
maybe they heard my voice :joy:

1 Like