Upgrade script fails at DB upgrade step due to collections.abc module

Issue Summary

Hi i am running upgrade script which is failing either on ubuntu 16.04 LTS and 20.04 LTS
there are no redash process running and pg_stat_activity is also checked and empty.

sudo bin/upgrade

Failed running: sudo -u redash bin/run ./manage.py db upgrade
Exit status: 1
Output:
Traceback (most recent call last):
File “/opt/redash/redash.4.0.1.b4038/manage.py”, line 6, in
from redash.cli import manager
File “/opt/redash/redash.4.0.1.b4038/redash/init.py”, line 72, in
import_query_runners(settings.QUERY_RUNNERS)
File “/opt/redash/redash.4.0.1.b4038/redash/query_runner/init.py”, line 179, in import_query_runners
import(runner_import)
File “/opt/redash/redash.4.0.1.b4038/redash/query_runner/query_results.py”, line 9, in
from redash import models
File “/opt/redash/redash.4.0.1.b4038/redash/models.py”, line 38, in
from sqlalchemy_searchable import SearchQueryMixin, make_searchable, vectorizer
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy_searchable/init.py”, line 9, in
from sqlalchemy_utils import TSVectorType
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy_utils/init.py”, line 1, in
from .aggregates import aggregated # noqa
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy_utils/aggregates.py”, line 372, in
from .functions.orm import get_column_key
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy_utils/functions/init.py”, line 1, in
from .database import ( # noqa
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy_utils/functions/database.py”, line 3, in
from collections.abc import Mapping, Sequence
ImportError: No module named abc

Maybe some library/module to upgrade in the script ?
Your help will be much appreciated !

Many thanks for this bold app @arikfr !!

Technical details:

  • Redash Version: Upgrading V3 to V4
  • Browser/OS: Ubuntu 16.04 & Ubuntu 20.04 LTS
  • How did you install Redash: following the legacy guide
1 Like

I experience the same issue, but trying to upgrade from 1.0.3 to 5.0.1

It seems to be an issue with python versions: python - Cannot import 'collections.abc' when using windows subsystem for linux - Stack Overflow

EDIT: it’s actually a pip dependency… read below

1 Like

Ok actually it seems it’s a pip dependency issue.

I just checked my requirements.txt and found this:
SQLAlchemy-Utils>=0.29.0

It’s the only one that doesn’t match an exact version, so it must be now installing a recent version that’s not backwards compatible with python 2.

Try changing that package to an exact version, like SQLAlchemy-Utils==0.32.21 . Note it’s gotta be the requirements file under the version you are installing (v4).

You might have to edit the script or run some things manually…
I think re-running the upgrade script might overwrite the requirements file even if you edit it first. See if it works, if not then maybe install exact version manually using pip install, and repeat. If it still fails, you’ll have to get creative. :sweat_smile:

EDIT: 0.29 is extremely old (2015) so try 0.32.21 which was released around the same time (end of 2017) as the other sql-alchemy packages in the requirements file.

1 Like

I’ve checked the SQLAlchemy-Utils version installed on my instance :

$ pip show SQLAlchemy-Utils
Name: SQLAlchemy-Utils
Version: 0.36.8
Summary: Various utility functions for SQLAlchemy.
Home-page: https://github.com/kvesteri/sqlalchemy-utils
Author: Konsta Vesterinen, Ryan Leckey, Janne Vanhala, Vesa Uimonen
Author-email: konsta@fastmonkeys.com
License: BSD
Location: /usr/local/lib/python2.7/dist-packages
Requires: six, SQLAlchemy

EDIT : I’ve downgraded my SQLAlchemy-Utils to 0.32.21 and the upgrade script finished successfully :slight_smile:

1 Like

indeed downgrading it to version 0.32.21 worked for me too.
Many thanks @androsj you are a true legend here !!!