1

Issue Summary

redash datasource connection fails after redash running for a few days

Technical details:

We are running redash using EKS. After the application runs for several days, the datasources become reachable. Normally we can get the connections back after we restart the redash_adhocworker. But today, I cannot get any datasource connection back after I restarted the whole stack

  • Redash Version: 11.0.0-dev
  • Browser/OS: latest chrome or firefox
  • How did you install Redash: on EKS

The Datasource connection test fails with error:

Connection Test Failed:
Parameter validation failed: Unknown parameter in QueryExecutionContext: “Catalog”, must be one of: Database

I am not sure what parameter it is complaining about. I really appreciate it If anyone can give any suggestion.

The redash adhocworker node log:

[2022-02-09 21:05:52,698][PID:567][DEBUG][botocore.loaders] Loading JSON file: /usr/local/lib/python3.7/site-packages/botocore/data/_retry.json
[2022-02-09 21:05:52,699][PID:567][DEBUG][botocore.client] Registering retry handlers for service: athena
[2022-02-09 21:05:52,701][PID:567][DEBUG][pyathena.common] SELECT 1
[2022-02-09 21:05:52,702][PID:567][DEBUG][botocore.hooks] Event before-parameter-build.athena.StartQueryExecution: calling handler <function generate_idempotent_uuid at 0x7f48866d7b90>
[2022-02-09 21:05:52,702][PID:567][DEBUG][botocore.handlers] injecting idempotency token (e4beee30-0f20-43e3-843a-69ba2a5ff728) into param 'ClientRequestToken'.
[2022-02-09 21:05:52,702][PID:567][ERROR][pyathena.common] Failed to execute query.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pyathena/common.py", line 417, in _execute
    **request
  File "/usr/local/lib/python3.7/site-packages/pyathena/util.py", line 84, in retry_api_call
    return retry(func, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/tenacity/__init__.py", line 404, in __call__
    do = self.iter(retry_state=retry_state)
  File "/usr/local/lib/python3.7/site-packages/tenacity/__init__.py", line 349, in iter
    return fut.result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
    return self.__get_result()
  File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.7/site-packages/tenacity/__init__.py", line 407, in __call__
    result = fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 276, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 559, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 607, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/lib/python3.7/site-packages/botocore/validate.py", line 297, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Unknown parameter in QueryExecutionContext: "Catalog", must be one of: Database
[2022-02-09 21:05:52,756][PID:567][INFO][rq.worker] queries: Job OK (0491b875-2751-44cc-a4a2-6a5a50a66e56)
1 Like
2

I’ve found the same issue.
As I explored some codes, this seems to be caused by PyAthena, not Redash.

I guess this trouble is occurred in here, and I was able to get around it by changing specified version of PyAthena as below.

PyAthena>=1.5.0,<=1.11.5

I’m going to seek more efficient solution or create Pull Request for quick fix.

I hope this is helpful for you.

2 Likes
3

Thanks for posting this. We’re happy to review a Pull Request implementing a fix if you’re so inclined :pray:

4

:+1: i hereby confirm this works! without it, PyAthena would be upgraded to the latest version causing that error

#15 8.298 Collecting PyAthena>=1.5.0
#15 8.302   Downloading pyathena-2.21.0-py3-none-any.whl (67 kB)
5