I am setting up on premises Redash into Kubernetes cluster (hosted in AWS) with the latest docker images.
I tried the docker-compose.porduction.yaml with docker and it worked fine.
Then I made Kubernetes services and deployment yaml configurations for server, worker and redis and configured the DB.
Create_db job worked fine, but if I use the Redash UI, then it is very slow (page loads about a minute) … but at the end it loads correctly.
I believe I have something misconfigured.
From server logs I see long “query_duration” but no errors:
[2017-05-03 09:02:13,233][PID:32180][INFO][metrics] method=GET path=/api/queries/recent endpoint=recent_queries status=200 content_type=application/json content_length=2 duration=119.76 query_count=5 query_duration=101.95 [2017-05-03 09:02:33,237][PID:32556][DEBUG][metrics] table=organizations query=select duration=2.41
From server logs I also see that workers are getting timeout regularly:
[2017-05-03 11:56:57 +0000]  [CRITICAL] WORKER TIMEOUT (pid:11622) [2017-05-03 11:56:57 +0000]  [INFO] Worker exiting (pid: 11622) [2017-05-03 11:56:58 +0000]  [INFO] Booting worker with pid: 11872
From the DB logs the query times are less than 1ms.
2017-05-03 11:28:26.783 UTC : [20-1] user=redash,db=redash,client=10.24.221.104(43642),app=[unknown] LOG: duration: 0.308 ms statement: SELECT users.groups AS users_groups, users.updated_at AS users_updated_at, users.created_at AS users_created_at, users.id AS users_id, users.org_id AS users_org_id, users.name AS users_name, users.email AS users_email, users.password_hash AS users_password_hash, users.api_key AS users_api_key
From the DB logs I also see that the client is closing connection regularly … might be related worker timeouts:
2017-05-03 02:45:00.327 UTC : [1-1] user=redash,db=redash,client=10.44.221.236(40460),app=[unknown] LOG: could not receive data from client: Connection reset by peer
Any suggestion about how get the redash UI normal.
Is there any diagram about how the server and workers are communicating and about Redash architecture?