I’m having problems updating Redash 6.0.0.b8537 to 7.0.0.b18042.
I’m hosting it on Kubernetes (Google Kubernetes Engine) and using Google Cloud SQL as a backend. Due to Kubernetes, the normal update instructions won’t work because docker-compose is not used.
Updating Redash itself works, but I can’t get db migrate to work.
I tried to recreate update steps in the following way:
- Snapshot Cloud SQL for backup
- Delete Redash deployment (I chose to delete whole deployment because trying to kill pods/containers/processes inside turned out very tricky/impossible). At this point, Redash services are practically turned off and DB is still running.
- Run Kubernetes job to migrate DB. This will launch Redash container (version 7.0) and only run
python manage.py db upgrade-command and exits after that.
Migrate script gives following output before completing:
[2019-04-02 13:33:29,826][PID:6][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2019-04-02 13:33:29,912][PID:6][INFO][root] Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2019-04-02 13:33:41,363][PID:6][INFO][alembic.runtime.migration] Context impl PostgresqlImpl.
[2019-04-02 13:33:41,365][PID:6][INFO][alembic.runtime.migration] Will assume transactional DDL.
- Re-deploy Redash services using the 7.0 version
Trying to access web interface after upgrade results in Internal Server Error.
Logs are referring to error in DB schema:
[2019-04-02 13:36:31,510][PID:9][ERROR][redash] Exception on /favicon.ico [GET]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1636, in full_dispatch_request
request_started.send(self)
File "/usr/local/lib/python2.7/dist-packages/blinker/base.py", line 267, in send
for receiver in self.receivers_for(sender)]
File "/app/redash/models/users.py", line 54, in update_user_active_at
if current_user.is_authenticated and not current_user.is_api_user():
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 343, in __getattr__
return getattr(self._get_current_object(), name)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 302, in _get_current_object
return self.__local()
File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 26, in <lambda>
current_user = LocalProxy(lambda: _get_user())
File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 302, in _get_user
current_app.login_manager._load_user()
File "/usr/local/lib/python2.7/dist-packages/flask_login/login_manager.py", line 317, in _load_user
return self.reload_user()
File "/usr/local/lib/python2.7/dist-packages/flask_login/login_manager.py", line 279, in reload_user
user = self.user_callback(user_id)
File "/app/redash/authentication/__init__.py", line 63, in load_user
user = models.User.get_by_id_and_org(user_id, org)
File "/app/redash/models/mixins.py", line 28, in get_by_id_and_org
return query.one()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2954, in one
ret = self.one_or_none()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2924, in one_or_none
ret = list(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2995, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 3018, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (psycopg2.ProgrammingError) column users.details does not exist
LINE 1: ..._api_key, users.disabled_at AS users_disabled_at, users.deta...
^
HINT: Perhaps you meant to reference the column "users.email".
[SQL: 'SELECT users.profile_image_url AS users_profile_image_url, users.groups AS users_groups, users.updated_at AS users_updated_at, users.created_at AS users_created_at, users.id AS users_id, users.org_id AS users_org_id, users.name AS users_name, users.email AS users_email, users.password_hash AS users_password_hash, users.api_key AS users_api_key, users.disabled_at AS users_disabled_at, users.details AS users_details \nFROM users \nWHERE users.id = %(id_1)s AND %(param_1)s = users.org_id'] [parameters: {'id_1': u'1', 'param_1': 1}] (Background on this error at: http://sqlalche.me/e/f405)
To me it looks like DB migrate doesn’t run as it should. I’ve also tried to run it directly from deployed Redash server without shutting down the deployment, but it still gives the same output.
perhaps there was some kind of conflict/error somewhere in deployment with version tags that was accidentally fixed.
