Upgrade from v6.0.0.b8537 to v7.0.0.b18042 broke saml and gives 500 internal server error

#1

Issue Summary

I am using Redash native docker based setup and tried to upgrade from v6 to v7 today and had the same issue. SAML OKTA broke and gave me an error 500 internal server error.
I am using the okta URL https://xxxxxxxxxxxxxxx/saml/callback?org_slug=default
I am using external RDS database , so I didn’t run the db upgrade command during the upgrade process.

Upgrade Process followed per documentation

  1. Make sure to backup your data. You only need to backup Redash’s PostgreSQL database (the database Redash stores metadata in, not the ones you might be querying) as the data in Redis is transient.
  2. Update /opt/redash/docker-compose.yml Redash image reference to the one you want to upgrade to.
  3. Stop Redash services: docker-compose stop server scheduler scheduled_worker adhoc_worker (you might need to list additional services if you updated your configuration)
    ####Didn’t do the manage db upgrade step below because I am assuming we don’t need it when using RDS postgres backend. Correct me if I am wrong.
  4. Apply migration (if necessary): docker-compose run --rm server manage db upgrade
  5. Start services: docker-compose up -d

I got the error in container logs:
[2019-03-26 18:19:40,139][PID:126][ERROR][redash] Exception on /saml/callback [POST]
Traceback (most recent call last):
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1988, in wsgi_app
response = self.full_dispatch_request()
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/local/lib/python2.7/dist-packages/flask_restful/ init .py”, line 271, in error_router
return original_handler(e)
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1639, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1625, in dispatch_request
return self.view_functionsrule.endpoint
File “/app/redash/authentication/saml_auth.py”, line 87, in idp_initiated
user = create_and_login_user(current_org, name, email)
File “/app/redash/authentication/ init .py”, line 259, in create_and_login_user
user_object = models.User.get_by_email_and_org(email, org)
File “/app/redash/models/users.py”, line 183, in get_by_email_and_org
return cls.get_by_org(org).filter(cls.email == email).one()
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2954, in one
ret = self.one_or_none()
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2924, in one_or_none
ret = list(self)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2995, in iter
return self._execute_and_instances(context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 3018, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 948, in execute
return meth(self, multiparams, params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py”, line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1200, in _execute_context
context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1413, in _handle_dbapi_exception
exc_info
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py”, line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1193, in _execute_context
context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py”, line 509, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (psycopg2.ProgrammingError) column users.details does not exist
LINE 1: …_api_key, users.disabled_at AS users_disabled_at, users.deta…
^
HINT: Perhaps you meant to reference the column “users.email”.
[SQL: ‘SELECT users.profile_image_url AS users_profile_image_url, users.groups AS users_groups, users.updated_at AS users_updated_at, users.created_at AS users_created_at, users.id AS users_id, users.org_id AS users_org_id, users.name AS users_name, users.email AS users_email, users.password_hash AS users_password_hash, users.api_key AS users_api_key, users.disabled_at AS users_disabled_at, users.details AS users_details \nFROM users \nWHERE %(param_1)s = users.org_id AND users.email = lower(%(lower_1)s)’] [parameters: {‘lower_1’: ‘nagaraju.balusa@disqo.com’, ‘param_1’: 1}] (Background on this error at: http://sqlalche.me/e/f405)
[2019-03-26 18:19:40,140][PID:126][INFO][metrics] method=POST path=/saml/callback endpoint=saml_auth_idp_initiated status=500 content_type=? content_length=-1 duration=182.67 query_count=1 query_duration=3.20

When I rollback to v6 the saml works fine and authenticates me.
Does anyone have issues upgrading to v7 too.?
Can anyone help if they did resolve this issue?

Technical details:

  • Redash Version: v6.0.0.b8537 to v7.0.0.b18042
  • Browser/OS: safari and chrome
  • How did you install Redash: docker based images setup.sh script, rds backend url

docker compose file used: #### I am using /redash as my directory becuse it is a mount and backed up daily.

version: ‘2’
x-redash-service: &redash-service
image: redash/redash:6.0.0.b8537
depends_on:
- postgres
- redis
env_file: /redash/env
restart: always
services:
server:
<<: *redash-service
command: server
ports:
- “5000:5000”
environment:
REDASH_WEB_WORKERS: 4
scheduler:
<<: *redash-service
command: scheduler
environment:
QUEUES: “celery”
WORKERS_COUNT: 1
scheduled_worker:
<<: *redash-service
command: worker
environment:
QUEUES: “scheduled_queries,schemas”
WORKERS_COUNT: 1
adhoc_worker:
<<: *redash-service
command: worker
environment:
QUEUES: “queries”
WORKERS_COUNT: 2
redis:
image: redis:3.0-alpine
restart: always
postgres:
image: postgres:9.5.6-alpine
env_file: /redash/env
volumes:
- /redash/postgres-data:/var/lib/postgresql/data
restart: always
nginx:
image: redash/nginx:latest
ports:
- “80:80”
depends_on:
- server
links:
- server:redash
restart: always

0 Likes

#2

Based on the error message, it looks like you didn’t apply migrations. It’s not related to SAML, it just the users table missing a column.

0 Likes

#3

It worked thanks, I assumed the documentation said if necessary do the migration.
It would be nice to see more documentation on RDS backend usage and the docker compose modifications with that setup respectively.

0 Likes

#4

It’s basically:

  1. Move data to RDS.
  2. Update the DATABASE_URL setting in /opt/redash/env file to the URL of your RDS instance.
  3. Remove the postgres container from docker-compose.yml.
  4. Run docker-compose up -d.
0 Likes

#5

Thanks for the update.

0 Likes