Hi, I have been using Redash for a while now on my Kubernetes environment. But sometimes I keep getting “Internal Server Error” when I try to open my redash page but when I refresh the page a few times, the page will load normally without the error again. Does anybody experience this before?
This doesn’t sound normal. Can you share any information about the failures? Your logs? Have you restarted everything?
As K4s1m was saying without any logs it will be complicated; I faced same kind of issue solved by SQLALCHEMY_ENABLE_POOL_PRE_PING: “true” but it could be completely something else …
Thanks k4s1m & xavier-d for the responses.
Here is the log when this event occured. When I first access the page, it returned the Internal Server Error message but as I refresh the page, it successfully load the login page
[2021-04-17 05:39:56,523][PID:919][ERROR][redash] Exception on /login [GET]
Traceback (most recent call last):
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1988, in wsgi_app
response = self.full_dispatch_request()
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1641, in full_dispatch_request
rv = self.handle_user_exception(e)
File “/usr/local/lib/python2.7/dist-packages/flask_restful/init.py”, line 271, in error_router
return original_handler(e)
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1544, in handle_user_exception
reraise(exc_type, exc_value, tb)
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1639, in full_dispatch_request
rv = self.dispatch_request()
File “/usr/local/lib/python2.7/dist-packages/flask/app.py”, line 1625, in dispatch_request
return self.view_functionsrule.endpoint
File “/usr/local/lib/python2.7/dist-packages/flask_limiter/extension.py”, line 442, in __inner
return obj(*a, **k)
File “/app/redash/handlers/authentication.py”, line 143, in login
if current_org == None and not settings.MULTI_ORG:
File “/usr/local/lib/python2.7/dist-packages/werkzeug/local.py”, line 365, in
eq = lambda x, o: x._get_current_object() == o
File “/usr/local/lib/python2.7/dist-packages/werkzeug/local.py”, line 302, in _get_current_object
return self.__local()
File “/app/redash/authentication/org_resolving.py”, line 18, in _get_current_org
g .org = Organization.get_by_slug(slug)
File “/app/redash/models/organizations.py”, line 33, in get_by_slug
return cls.query.filter(cls.slug == slug).first()
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2895, in first
ret = list(self[0:1])
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2687, in getitem
return list(res)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 2995, in iter
return self._execute_and_instances(context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py”, line 3018, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 948, in execute
return meth(self, multiparams, params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py”, line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1200, in _execute_context
context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1413, in _handle_dbapi_exception
exc_info
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py”, line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py”, line 1193, in _execute_context
context)
File “/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py”, line 509, in do_execute
cursor.execute(statement, parameters)
OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: ‘SELECT organizations.updated_at AS organizations_updated_at, organizations.created_at AS organizations_created_at, organizations.id AS organizations_id, organizations.name AS organizations_name, organizations.slug AS organizations_slug, organizations.settings AS organizations_settings \nFROM organizations \nWHERE organizations.slug = %(slug_1)s \n LIMIT %(param_1)s’] [parameters: {‘slug_1’: ‘default’, ‘param_1’: 1}] (Background on this error at: http://sqlalche.me/e/e3q8 )
[2021-04-17 05:39:56,527][PID:919][INFO][metrics] method=GET path=/login endpoint=redash_login status=500 content_type=? content_length=-1 duration=10.03 query_count=0 query_duration=0.00
[2021-04-17 05:39:59,427][PID:1121][INFO][metrics] method=GET path=/login endpoint=redash_login status=200 content_type=text/html; charset=utf-8 content_length=1827 duration=29.98 query_count=1 query_duration=4.42
I have added “SQLALCHEMY_ENABLE_POOL_PRE_PING”: true on the environment variable but I’m still getting this error. Currently I’m using Redash version 8.0.2.b37747 that was pulled from docker registry.
Hi, I have solved the issue by using this solution and applying it to /app/redash/models/base.py
"OperationalError: server closed the connection unexpectedly" on SQL: ‘SELECT … FROM organizations’ · Issue #4675 · getredash/redash · GitHub.
Apparently the docker version of Redash does not have pool_pre_ping option so I have to mount it manually to the deployment.
Thank you for the helps.
3 Likes