Refresh Schedule does not work correctly

saito-ya · January 29, 2022, 6:43am

Issue Summary

Hello,

I construct Redash environment on AWS.
I am using ECS to run redash server, worker and scheduler.
The followings are the summary of the task definition for scheduler container.

image: ‘redash/redash:10.1.0.b50633’
command: ‘scheduler’
environment variables
- PYTHONUNBUFFERED: ‘0’,
- REDASH_REDIS_URL: redisUrl,
- REDASH_DATABASE_URL: dbUrl,
- REDASH_COOKIE_SECRET: secret,
- REDASH_LOG_LEVEL: ‘INFO’,
- REDASH_DATE_FORMAT: ‘YYYY/MM/DD’,
- QUEUES: ‘celery’,
- WORKERS_COUNT: ‘2’

However, the scheduled query have not been executed.
I got the following logs for scheduler task, but I can not find the error log.

2022-01-29T15:28:24.449+09:00	Starting RQ scheduler...
2022-01-29T15:28:27.686+09:00	[2022-01-29 06:28:27,686][PID:1][INFO][rq_scheduler.scheduler] Registering birth

Could you please tell me how to set up to work refresh schedule?

Thank you in advance.

Technical details:

Redash Version: v10 (image tag: 10.1.0.b50633)
Browser/OS: Chorme / Mac OS
How did you install Redash: Deploy server, worker and scheduler to ECS service using redash docker image.

jesse · February 1, 2022, 3:28pm

Do you have any workers monitoring the scheduled_queries queue?

You can check this as an admin by visiting the System Status page, under the Workers tab:

saito-ya · February 2, 2022, 2:04pm

Hi jesse!

Thank you for your reply!

I checked Workers and could find scheduled_queries as one of queues.
Could you please tell me other place to check?

Thank you inadvance.

saito-ya · February 2, 2022, 2:50pm

FYI

I added default and periodic to Queues as well as scheduled_queries.
However scheduled queries are not executed.

jesse · February 2, 2022, 6:10pm

How many instances of redis are you using in your setup? Have you confirmed that absolutely no queries are executing on a refresh schedule?

saito-ya · February 3, 2022, 12:58pm

Thank you for your reply!

How many instances of redis are you using in your setup?

one instance (node).

Have you confirmed that absolutely no queries are executing on a refresh schedule?

Sorry, I don’t know exact way to confirm that no queries are executed on a refresh schedule.
However, I can see rq.job.redash.tasks.queries.execution logs and its queue value is queries not scheduled_queries.

If there is correct way to confirm whether queries are executed on a refresh schedule, could you please tell me that?

Thank you in advance!

jesse · February 3, 2022, 1:02pm

Sure, just write a sinple query. Give it a one minute refresh schedule. Restart your Redash services. And see if it executed periodically.

saito-ya · February 3, 2022, 1:20pm

Thank you for your quick reply!

just write a sinple query. Give it a one minute refresh schedule. Restart your Redash services. And see if it executed periodically.

I tried that.
However the last executed time of the query is not been changed.
Refresh Schedule doesn’t work yet.

I will atach log file of worker service.
Could you please check this and give some advice?

timestamp,message
1643894006552,Starting RQ worker...
1643894006758,"2022-02-03 13:13:26,758 INFO RPC interface 'supervisor' initialized"
1643894006758,"2022-02-03 13:13:26,758 CRIT Server 'unix_http_server' running without any HTTP authentication checking"
1643894006759,"2022-02-03 13:13:26,759 INFO supervisord started with pid 1"
1643894007761,"2022-02-03 13:13:27,761 INFO spawned: 'worker_healthcheck' with pid 10"
1643894007768,"2022-02-03 13:13:27,763 INFO spawned: 'worker-0' with pid 11"
1643894008735,"[2022-02-03 13:13:28,735][PID:10][DEBUG][redash.query_runner] Registering Amazon Athena (athena) query runner."
1643894008735,"[2022-02-03 13:13:28,735][PID:11][DEBUG][redash.query_runner] Registering Amazon Athena (athena) query runner."
1643894008869,"[2022-02-03 13:13:28,869][PID:10][DEBUG][redash.query_runner] Registering BigQuery (bigquery) query runner."
1643894008869,"[2022-02-03 13:13:28,869][PID:11][DEBUG][redash.query_runner] Registering BigQuery (bigquery) query runner."
1643894008869,"2022-02-03 13:13:28,869 INFO success: worker_healthcheck entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)"
1643894008893,"[2022-02-03 13:13:28,892][PID:10][DEBUG][redash.query_runner] Registering Google Sheets (google_spreadsheets) query runner."
1643894008893,"[2022-02-03 13:13:28,893][PID:11][DEBUG][redash.query_runner] Registering Google Sheets (google_spreadsheets) query runner."
1643894008894,"[2022-02-03 13:13:28,894][PID:10][DEBUG][redash.query_runner] Registering Graphite (graphite) query runner."
1643894008895,"[2022-02-03 13:13:28,895][PID:11][DEBUG][redash.query_runner] Registering Graphite (graphite) query runner."
1643894008945,"[2022-02-03 13:13:28,945][PID:10][DEBUG][redash.query_runner] Registering MongoDB (mongodb) query runner."
1643894008948,"[2022-02-03 13:13:28,948][PID:11][DEBUG][redash.query_runner] Registering MongoDB (mongodb) query runner."
1643894008948,"[2022-02-03 13:13:28,948][PID:10][DEBUG][redash.query_runner] Registering Couchbase (couchbase) query runner."
1643894008949,"[2022-02-03 13:13:28,948][PID:11][DEBUG][redash.query_runner] Registering Couchbase (couchbase) query runner."
1643894008958,"[2022-02-03 13:13:28,958][PID:10][DEBUG][redash.query_runner] Registering MySQL (mysql) query runner."
1643894008958,"[2022-02-03 13:13:28,958][PID:11][DEBUG][redash.query_runner] Registering MySQL (mysql) query runner."
1643894008958,"[2022-02-03 13:13:28,958][PID:11][DEBUG][redash.query_runner] Registering MySQL (Amazon RDS) (rds_mysql) query runner."
1643894008958,"[2022-02-03 13:13:28,958][PID:10][DEBUG][redash.query_runner] Registering MySQL (Amazon RDS) (rds_mysql) query runner."
1643894008979,"[2022-02-03 13:13:28,979][PID:11][DEBUG][redash.query_runner] Registering PostgreSQL (pg) query runner."
1643894008979,"[2022-02-03 13:13:28,979][PID:11][DEBUG][redash.query_runner] Registering Redshift (redshift) query runner."
1643894008979,"[2022-02-03 13:13:28,979][PID:11][DEBUG][redash.query_runner] Registering Redshift (with IAM User/Role) (redshift_iam) query runner."
1643894008979,"[2022-02-03 13:13:28,979][PID:11][DEBUG][redash.query_runner] Registering CockroachDB (cockroach) query runner."
1643894008979,"[2022-02-03 13:13:28,979][PID:10][DEBUG][redash.query_runner] Registering PostgreSQL (pg) query runner."
1643894008980,"[2022-02-03 13:13:28,980][PID:10][DEBUG][redash.query_runner] Registering Redshift (redshift) query runner."
1643894008980,"[2022-02-03 13:13:28,980][PID:10][DEBUG][redash.query_runner] Registering Redshift (with IAM User/Role) (redshift_iam) query runner."
1643894008980,"[2022-02-03 13:13:28,980][PID:11][DEBUG][redash.query_runner] Registering Url (url) query runner."
1643894008980,"[2022-02-03 13:13:28,980][PID:10][DEBUG][redash.query_runner] Registering CockroachDB (cockroach) query runner."
1643894008981,"[2022-02-03 13:13:28,981][PID:10][DEBUG][redash.query_runner] Registering Url (url) query runner."
1643894009443,"[2022-02-03 13:13:29,443][PID:10][DEBUG][redash.query_runner] Registering InfluxDB (influxdb) query runner."
1643894009448,"[2022-02-03 13:13:29,448][PID:11][DEBUG][redash.query_runner] Registering InfluxDB (influxdb) query runner."
1643894009449,"[2022-02-03 13:13:29,449][PID:10][DEBUG][redash.query_runner] Registering Kibana (kibana) query runner."
1643894009449,"[2022-02-03 13:13:29,449][PID:10][DEBUG][redash.query_runner] Registering Elasticsearch (elasticsearch) query runner."
1643894009451,"[2022-02-03 13:13:29,451][PID:10][DEBUG][redash.query_runner] Registering Amazon Elasticsearch Service (aws_es) query runner."
1643894009453,"[2022-02-03 13:13:29,453][PID:11][DEBUG][redash.query_runner] Registering Kibana (kibana) query runner."
1643894009453,"[2022-02-03 13:13:29,453][PID:11][DEBUG][redash.query_runner] Registering Elasticsearch (elasticsearch) query runner."
1643894009454,"[2022-02-03 13:13:29,454][PID:11][DEBUG][redash.query_runner] Registering Amazon Elasticsearch Service (aws_es) query runner."
1643894009458,"[2022-02-03 13:13:29,458][PID:10][DEBUG][redash.query_runner] Registering Trino (trino) query runner."
1643894009458,"[2022-02-03 13:13:29,458][PID:11][DEBUG][redash.query_runner] Registering Trino (trino) query runner."
1643894009468,"[2022-02-03 13:13:29,467][PID:11][DEBUG][redash.query_runner] Registering Presto (presto) query runner."
1643894009468,"[2022-02-03 13:13:29,468][PID:10][DEBUG][redash.query_runner] Registering Presto (presto) query runner."
1643894009471,"[2022-02-03 13:13:29,471][PID:11][DEBUG][redash.query_runner] Registering Databricks (databricks) query runner."
1643894009472,"[2022-02-03 13:13:29,472][PID:10][DEBUG][redash.query_runner] Registering Databricks (databricks) query runner."
1643894009485,"[2022-02-03 13:13:29,485][PID:11][DEBUG][redash.query_runner] Registering Hive (hive) query runner."
1643894009485,"[2022-02-03 13:13:29,485][PID:11][DEBUG][redash.query_runner] Registering Hive (HTTP) (hive_http) query runner."
1643894009486,"[2022-02-03 13:13:29,486][PID:10][DEBUG][redash.query_runner] Registering Hive (hive) query runner."
1643894009487,"[2022-02-03 13:13:29,486][PID:10][DEBUG][redash.query_runner] Registering Hive (HTTP) (hive_http) query runner."
1643894010449,"[2022-02-03 13:13:30,448][PID:10][DEBUG][redash.query_runner] Registering Impala (impala) query runner."
1643894010453,"[2022-02-03 13:13:30,453][PID:11][DEBUG][redash.query_runner] Registering Impala (impala) query runner."
1643894010480,"[2022-02-03 13:13:30,480][PID:10][DEBUG][redash.query_runner] Registering Vertica (vertica) query runner."
1643894010482,"[2022-02-03 13:13:30,482][PID:10][DEBUG][redash.query_runner] Registering ClickHouse (clickhouse) query runner."
1643894010484,"[2022-02-03 13:13:30,484][PID:11][DEBUG][redash.query_runner] Registering Vertica (vertica) query runner."
1643894010484,"[2022-02-03 13:13:30,484][PID:11][DEBUG][redash.query_runner] Registering ClickHouse (clickhouse) query runner."
1643894010509,"[2022-02-03 13:13:30,509][PID:10][DEBUG][redash.query_runner] Registering Yandex Metrica (yandex_metrika) query runner."
1643894010509,"[2022-02-03 13:13:30,509][PID:10][DEBUG][redash.query_runner] Registering Yandex AppMetrica (yandex_appmetrika) query runner."
1643894010510,"[2022-02-03 13:13:30,510][PID:11][DEBUG][redash.query_runner] Registering Yandex Metrica (yandex_metrika) query runner."
1643894010511,"[2022-02-03 13:13:30,511][PID:11][DEBUG][redash.query_runner] Registering Yandex AppMetrica (yandex_appmetrika) query runner."
1643894010511,"[2022-02-03 13:13:30,511][PID:10][DEBUG][redash.query_runner] Registering Rockset (rockset) query runner."
1643894010511,"[2022-02-03 13:13:30,511][PID:11][DEBUG][redash.query_runner] Registering Rockset (rockset) query runner."
1643894010522,"[2022-02-03 13:13:30,522][PID:10][DEBUG][redash.query_runner] Registering TreasureData (treasuredata) query runner."
1643894010523,"[2022-02-03 13:13:30,523][PID:11][DEBUG][redash.query_runner] Registering TreasureData (treasuredata) query runner."
1643894010526,"[2022-02-03 13:13:30,526][PID:10][DEBUG][redash.query_runner] Registering Sqlite (sqlite) query runner."
1643894010526,"[2022-02-03 13:13:30,526][PID:11][DEBUG][redash.query_runner] Registering Sqlite (sqlite) query runner."
1643894010594,"[2022-02-03 13:13:30,594][PID:11][DEBUG][redash.query_runner] Registering DynamoDB (with DQL) (dynamodb_sql) query runner."
1643894010594,"[2022-02-03 13:13:30,594][PID:10][DEBUG][redash.query_runner] Registering DynamoDB (with DQL) (dynamodb_sql) query runner."
1643894010599,"[2022-02-03 13:13:30,599][PID:10][DEBUG][redash.query_runner] Registering Microsoft SQL Server (mssql) query runner."
1643894010599,"[2022-02-03 13:13:30,599][PID:11][DEBUG][redash.query_runner] Registering Microsoft SQL Server (mssql) query runner."
1643894010600,"[2022-02-03 13:13:30,600][PID:10][DEBUG][redash.query_runner] Registering Microsoft SQL Server (ODBC) (mssql_odbc) query runner."
1643894010601,"[2022-02-03 13:13:30,601][PID:11][DEBUG][redash.query_runner] Registering Microsoft SQL Server (ODBC) (mssql_odbc) query runner."
1643894010603,"[2022-02-03 13:13:30,603][PID:10][DEBUG][redash.query_runner] Registering MemSQL (memsql) query runner."
1643894010604,"[2022-02-03 13:13:30,604][PID:11][DEBUG][redash.query_runner] Registering MemSQL (memsql) query runner."
1643894010607,"[2022-02-03 13:13:30,607][PID:10][DEBUG][thrift.transport.sslcompat] ipaddress module is available"
1643894010607,"[2022-02-03 13:13:30,607][PID:10][DEBUG][thrift.transport.sslcompat] ssl.match_hostname is available"
1643894010608,"[2022-02-03 13:13:30,608][PID:11][DEBUG][thrift.transport.sslcompat] ipaddress module is available"
1643894010608,"[2022-02-03 13:13:30,608][PID:11][DEBUG][thrift.transport.sslcompat] ssl.match_hostname is available"
1643894010638,"[2022-02-03 13:13:30,638][PID:10][DEBUG][redash.query_runner] Registering Mapd (mapd) query runner."
1643894010640,"[2022-02-03 13:13:30,640][PID:10][DEBUG][redash.query_runner] Registering JIRA (JQL) (jirajql) query runner."
1643894010641,"[2022-02-03 13:13:30,640][PID:11][DEBUG][redash.query_runner] Registering Mapd (mapd) query runner."
1643894010641,"[2022-02-03 13:13:30,641][PID:11][DEBUG][redash.query_runner] Registering JIRA (JQL) (jirajql) query runner."
1643894010643,"[2022-02-03 13:13:30,643][PID:10][DEBUG][redash.query_runner] Registering Google Analytics (google_analytics) query runner."
1643894010644,"[2022-02-03 13:13:30,644][PID:11][DEBUG][redash.query_runner] Registering Google Analytics (google_analytics) query runner."
1643894010660,"[2022-02-03 13:13:30,660][PID:10][DEBUG][root] Checking 'python-requests' version..."
1643894010661,"[2022-02-03 13:13:30,660][PID:11][DEBUG][root] Checking 'python-requests' version..."
1643894010661,"[2022-02-03 13:13:30,660][PID:11][DEBUG][root] Module 'python-requests' version is 2.21.0. The version is compatible."
1643894010661,"[2022-02-03 13:13:30,661][PID:11][DEBUG][redash.query_runner] Registering Axibase Time Series Database (axibasetsd) query runner."
1643894010662,"[2022-02-03 13:13:30,661][PID:10][DEBUG][root] Module 'python-requests' version is 2.21.0. The version is compatible."
1643894010662,"[2022-02-03 13:13:30,662][PID:10][DEBUG][redash.query_runner] Registering Axibase Time Series Database (axibasetsd) query runner."
1643894010690,"[2022-02-03 13:13:30,690][PID:11][DEBUG][redash.query_runner] Registering Salesforce (salesforce) query runner."
1643894010692,"[2022-02-03 13:13:30,691][PID:10][DEBUG][redash.query_runner] Registering Salesforce (salesforce) query runner."
1643894010745,"[2022-02-03 13:13:30,745][PID:11][DEBUG][passlib.utils.compat] loaded lazy attr 'SafeConfigParser': <class 'configparser.ConfigParser'>"
1643894010745,"[2022-02-03 13:13:30,745][PID:11][DEBUG][passlib.utils.compat] loaded lazy attr 'NativeStringIO': <class '_io.StringIO'>"
1643894010745,"[2022-02-03 13:13:30,745][PID:11][DEBUG][passlib.utils.compat] loaded lazy attr 'BytesIO': <class '_io.BytesIO'>"
1643894010746,"[2022-02-03 13:13:30,746][PID:10][DEBUG][passlib.utils.compat] loaded lazy attr 'SafeConfigParser': <class 'configparser.ConfigParser'>"
1643894010746,"[2022-02-03 13:13:30,746][PID:10][DEBUG][passlib.utils.compat] loaded lazy attr 'NativeStringIO': <class '_io.StringIO'>"
1643894010746,"[2022-02-03 13:13:30,746][PID:10][DEBUG][passlib.utils.compat] loaded lazy attr 'BytesIO': <class '_io.BytesIO'>"
1643894011512,"[2022-02-03 13:13:31,511][PID:10][DEBUG][redash.query_runner] Registering Query Results (results) query runner."
1643894011514,"[2022-02-03 13:13:31,514][PID:10][DEBUG][redash.query_runner] Registering Prometheus (prometheus) query runner."
1643894011524,"[2022-02-03 13:13:31,524][PID:11][DEBUG][redash.query_runner] Registering Query Results (results) query runner."
1643894011525,"[2022-02-03 13:13:31,524][PID:11][DEBUG][redash.query_runner] Registering Prometheus (prometheus) query runner."
1643894011548,"[2022-02-03 13:13:31,547][PID:10][DEBUG][redash.query_runner] Registering Qubole (qubole) query runner."
1643894011556,"[2022-02-03 13:13:31,556][PID:11][DEBUG][redash.query_runner] Registering Qubole (qubole) query runner."
1643894011557,"[2022-02-03 13:13:31,557][PID:10][DEBUG][redash.query_runner] Registering DB2 (db2) query runner."
1643894011560,"[2022-02-03 13:13:31,560][PID:10][DEBUG][redash.query_runner] Registering Druid (druid) query runner."
1643894011562,"[2022-02-03 13:13:31,562][PID:10][DEBUG][redash.query_runner] Registering Kylin (kylin) query runner."
1643894011563,"[2022-02-03 13:13:31,562][PID:11][DEBUG][redash.query_runner] Registering DB2 (db2) query runner."
1643894011564,"[2022-02-03 13:13:31,563][PID:10][DEBUG][redash.query_runner] Registering Apache Drill (drill) query runner."
1643894011564,"[2022-02-03 13:13:31,564][PID:11][DEBUG][redash.query_runner] Registering Druid (druid) query runner."
1643894011565,"[2022-02-03 13:13:31,565][PID:11][DEBUG][redash.query_runner] Registering Kylin (kylin) query runner."
1643894011565,"[2022-02-03 13:13:31,565][PID:11][DEBUG][redash.query_runner] Registering Apache Drill (drill) query runner."
1643894011568,"[2022-02-03 13:13:31,568][PID:10][DEBUG][redash.query_runner] Registering Uptycs (uptycs) query runner."
1643894011569,"[2022-02-03 13:13:31,569][PID:11][DEBUG][redash.query_runner] Registering Uptycs (uptycs) query runner."
1643894011698,"[2022-02-03 13:13:31,697][PID:10][DEBUG][snowflake.connector.ssl_wrap_socket] Injecting ssl_wrap_socket_with_ocsp"
1643894011700,"[2022-02-03 13:13:31,700][PID:10][DEBUG][snowflake.connector.auth] cache directory: /home/redash/.cache/snowflake"
1643894011702,"[2022-02-03 13:13:31,702][PID:11][DEBUG][snowflake.connector.ssl_wrap_socket] Injecting ssl_wrap_socket_with_ocsp"
1643894011704,"[2022-02-03 13:13:31,704][PID:11][DEBUG][snowflake.connector.auth] cache directory: /home/redash/.cache/snowflake"
1643894011919,"[2022-02-03 13:13:31,919][PID:10][DEBUG][redash.query_runner] Registering Snowflake (snowflake) query runner."
1643894011919,"[2022-02-03 13:13:31,919][PID:11][DEBUG][redash.query_runner] Registering Snowflake (snowflake) query runner."
1643894011942,"[2022-02-03 13:13:31,941][PID:11][DEBUG][redash.query_runner] Registering Phoenix (phoenix) query runner."
1643894011942,"[2022-02-03 13:13:31,942][PID:10][DEBUG][redash.query_runner] Registering Phoenix (phoenix) query runner."
1643894011944,"[2022-02-03 13:13:31,944][PID:11][DEBUG][redash.query_runner] Registering JSON (json) query runner."
1643894011945,"[2022-02-03 13:13:31,945][PID:10][DEBUG][redash.query_runner] Registering JSON (json) query runner."
1643894012000,"[2022-02-03 13:13:32,000][PID:11][DEBUG][redash.query_runner] Registering Cassandra (Cassandra) query runner."
1643894012000,"[2022-02-03 13:13:32,000][PID:11][DEBUG][redash.query_runner] Registering ScyllaDB (scylla) query runner."
1643894012001,"[2022-02-03 13:13:32,000][PID:10][DEBUG][redash.query_runner] Registering Cassandra (Cassandra) query runner."
1643894012001,"[2022-02-03 13:13:32,001][PID:10][DEBUG][redash.query_runner] Registering ScyllaDB (scylla) query runner."
1643894012027,"[2022-02-03 13:13:32,027][PID:10][DEBUG][redash.query_runner] Registering Dgraph (dgraph) query runner."
1643894012027,"[2022-02-03 13:13:32,027][PID:11][DEBUG][redash.query_runner] Registering Dgraph (dgraph) query runner."
1643894012039,"[2022-02-03 13:13:32,038][PID:11][DEBUG][redash.query_runner] Registering Azure Data Explorer (Kusto) (azure_kusto) query runner."
1643894012040,"[2022-02-03 13:13:32,039][PID:10][DEBUG][redash.query_runner] Registering Azure Data Explorer (Kusto) (azure_kusto) query runner."
1643894012050,"[2022-02-03 13:13:32,050][PID:11][DEBUG][redash.query_runner] Registering Exasol (exasol) query runner."
1643894012051,"[2022-02-03 13:13:32,051][PID:10][DEBUG][redash.query_runner] Registering Exasol (exasol) query runner."
1643894012052,"[2022-02-03 13:13:32,052][PID:11][DEBUG][redash.query_runner] Registering Amazon CloudWatch (cloudwatch) query runner."
1643894012053,"[2022-02-03 13:13:32,053][PID:11][DEBUG][redash.query_runner] Registering Amazon CloudWatch Logs Insights (cloudwatch_insights) query runner."
1643894012054,"[2022-02-03 13:13:32,054][PID:10][DEBUG][redash.query_runner] Registering Amazon CloudWatch (cloudwatch) query runner."
1643894012055,"[2022-02-03 13:13:32,054][PID:10][DEBUG][redash.query_runner] Registering Amazon CloudWatch Logs Insights (cloudwatch_insights) query runner."
1643894012212,"[2022-02-03 13:13:32,212][PID:11][DEBUG][redash.query_runner] Registering eccenca Corporate Memory (with SPARQL) (corporate_memory) query runner."
1643894012213,"[2022-02-03 13:13:32,212][PID:10][DEBUG][redash.query_runner] Registering eccenca Corporate Memory (with SPARQL) (corporate_memory) query runner."
1643894012216,"[2022-02-03 13:13:32,216][PID:10][DEBUG][redash.query_runner] Registering SPARQL Endpoint (sparql_endpoint) query runner."
1643894012216,"[2022-02-03 13:13:32,216][PID:11][DEBUG][redash.query_runner] Registering SPARQL Endpoint (sparql_endpoint) query runner."
1643894012403,"[2022-02-03 13:13:32,403][PID:10][DEBUG][redash.query_runner] Registering Excel (excel) query runner."
1643894012403,"[2022-02-03 13:13:32,403][PID:11][DEBUG][redash.query_runner] Registering Excel (excel) query runner."
1643894012405,"[2022-02-03 13:13:32,405][PID:10][DEBUG][redash.query_runner] Registering CSV (csv) query runner."
1643894012405,"[2022-02-03 13:13:32,405][PID:11][DEBUG][redash.query_runner] Registering CSV (csv) query runner."
1643894012409,"[2022-02-03 13:13:32,409][PID:10][DEBUG][redash.query_runner] Registering Firebolt (firebolt) query runner."
1643894012409,"[2022-02-03 13:13:32,409][PID:11][DEBUG][redash.query_runner] Registering Firebolt (firebolt) query runner."
1643894012410,"[2022-02-03 13:13:32,410][PID:10][DEBUG][redash.destinations] Registering Email (email) destinations."
1643894012411,"[2022-02-03 13:13:32,410][PID:11][DEBUG][redash.destinations] Registering Email (email) destinations."
1643894012411,"[2022-02-03 13:13:32,411][PID:10][DEBUG][redash.destinations] Registering Slack (slack) destinations."
1643894012412,"[2022-02-03 13:13:32,412][PID:11][DEBUG][redash.destinations] Registering Slack (slack) destinations."
1643894012440,"[2022-02-03 13:13:32,440][PID:10][DEBUG][redash.destinations] Registering Webhook (webhook) destinations."
1643894012441,"[2022-02-03 13:13:32,440][PID:11][DEBUG][redash.destinations] Registering Webhook (webhook) destinations."
1643894012442,"[2022-02-03 13:13:32,441][PID:10][DEBUG][redash.destinations] Registering HipChat (hipchat) destinations."
1643894012442,"[2022-02-03 13:13:32,442][PID:11][DEBUG][redash.destinations] Registering HipChat (hipchat) destinations."
1643894012443,"[2022-02-03 13:13:32,442][PID:10][DEBUG][redash.destinations] Registering Mattermost (mattermost) destinations."
1643894012443,"[2022-02-03 13:13:32,443][PID:11][DEBUG][redash.destinations] Registering Mattermost (mattermost) destinations."
1643894012444,"[2022-02-03 13:13:32,444][PID:10][DEBUG][redash.destinations] Registering ChatWork (chatwork) destinations."
1643894012444,"[2022-02-03 13:13:32,444][PID:11][DEBUG][redash.destinations] Registering ChatWork (chatwork) destinations."
1643894012451,"[2022-02-03 13:13:32,451][PID:10][DEBUG][redash.destinations] Registering PagerDuty (pagerduty) destinations."
1643894012451,"[2022-02-03 13:13:32,451][PID:11][DEBUG][redash.destinations] Registering PagerDuty (pagerduty) destinations."
1643894012452,"[2022-02-03 13:13:32,452][PID:10][DEBUG][redash.destinations] Registering Google Hangouts Chat (hangouts_chat) destinations."
1643894012452,"[2022-02-03 13:13:32,452][PID:11][DEBUG][redash.destinations] Registering Google Hangouts Chat (hangouts_chat) destinations."
1643894013281,"2022/02/03 13:13:33 [worker_healthcheck] Starting the health check for worker process Checks config: [(<class 'redash.cli.rq.WorkerHealthcheck'>, {})]"
1643894013281,2022/02/03 13:13:33 [worker_healthcheck] Installing signal handlers.
1643894013281,READY
1643894013360,"[2022-02-03 13:13:33,360][PID:11][INFO][rq.worker] Worker rq:worker:74693ce00ae247b6910f7cade78e605d: started, version 1.5.0"
1643894013362,"[2022-02-03 13:13:33,362][PID:11][INFO][rq.worker] *** Listening on default, queries, scheduled_queries, schemas, emails, periodic..."
1643894021966,"[2022-02-03 13:13:41,966][PID:11][INFO][rq.worker] default: cd545ffb-90ce-4694-bcbf-875d20730077"
1643894022047,"[2022-02-03 13:13:42,046][PID:22][DEBUG][metrics] table=events query=insert duration=5.76"
1643894022059,"[2022-02-03 13:13:42,058][PID:22][INFO][rq.worker] default: Job OK (cd545ffb-90ce-4694-bcbf-875d20730077)"
1643894022059,"[2022-02-03 13:13:42,058][PID:22][INFO][rq.worker] Result is kept for 500 seconds"
1643894041081,2022/02/03 13:14:01 [worker_healthcheck] Received TICK_60 event from supervisor
1643894041083,2022/02/03 13:14:01 [worker_healthcheck] No processes in state RUNNING found for process worker
1643894041083,RESULT 2
1643894041083,OKREADY
1643894096157,"[2022-02-03 13:14:56,156][PID:11][INFO][rq.worker] default: b46727f1-fa0f-4b70-8587-3edf73fc29d0"
1643894096237,"[2022-02-03 13:14:56,237][PID:23][DEBUG][metrics] table=events query=insert duration=5.80"
1643894096249,"[2022-02-03 13:14:56,249][PID:23][INFO][rq.worker] default: Job OK (b46727f1-fa0f-4b70-8587-3edf73fc29d0)"
1643894096250,"[2022-02-03 13:14:56,249][PID:23][INFO][rq.worker] Result is kept for 500 seconds"
1643894101254,2022/02/03 13:15:01 [worker_healthcheck] Received TICK_60 event from supervisor
1643894101255,RESULT 2
1643894101255,OKREADY
1643894101255,2022/02/03 13:15:01 [worker_healthcheck] No processes in state RUNNING found for process worker

jesse · February 3, 2022, 1:22pm

What queues does this worker monitor?

Edit: actually I can see what cues are being monitored here. I think your worker is fine. Now you need to check if there are any exceptions raised when your schedule runs.

saito-ya · February 3, 2022, 1:28pm

What queues does this worker monitor?

I set QUEUES: 'default,queries,scheduled_queries,schemas,emails,periodic' environment variable to worker service container.
So I think the worker is monitoring default, queries, scheduled_queries, schemas, emails and periodic.

Now you need to check if there are any exceptions raised when your schedule runs.

Actually I can’t see any exceptions in the log.

jesse · February 3, 2022, 1:33pm

what happens if you remove this line from your environment on the scheduler?

saito-ya · February 3, 2022, 1:41pm

I’m so sorry but let me update the information.
I don’t have scheduler service any more.

I have server service and worker servce.
These settings are bellow.

Server Service

image: “redash/redash:10.1.0.b50633”
command: “server”
environment variables
- PYTHONUNBUFFERED: ‘0’
- REDASH_HOST: ‘https://xxxxx.com’
- REDASH_REDIS_URL: redisUrl
- REDASH_DATABASE_URL: dbUrl
- REDASH_COOKIE_SECRET: ‘redash-cookie-secret’
- REDASH_LOG_LEVEL: ‘DEBUG’
- REDASH_DATE_FORMAT: ‘YYYY/MM/DD’

Worker Service

image: “redash/redash:10.1.0.b50633”
command: “worker”
environment variables
- PYTHONUNBUFFERED: ‘0’
- REDASH_HOST: ‘https://xxxxx.com’
- REDASH_REDIS_URL: redisUrl
- REDASH_DATABASE_URL: dbUrl
- REDASH_COOKIE_SECRET: ‘redash-cookie-secret’
- REDASH_LOG_LEVEL: ‘DEBUG’
- REDASH_DATE_FORMAT: ‘YYYY/MM/DD’
- QUEUES: ‘default,queries,scheduled_queries,schemas,emails,periodic’

So I don’t have celery in QUEUES right now.
Is container which execute scheduler command necessary?

jesse · February 3, 2022, 1:43pm

Well there’s your problem. If you don’t run a scheduler than your scheduled queries won’t execute.

Yes, of course it is. It wouldn’t be the default otherwise

saito-ya · February 3, 2022, 1:45pm

Yes, of course it is. It wouldn’t be the default otherwise

Sure, I will construct scheduler container.
Is QUEUES: celery environment variable necessary in scheduler container?

jesse · February 3, 2022, 1:46pm

No. You can look at our docker-compose for a sample configuration. The scheduler command already knows which queues to use so it does not require configuration.

saito-ya · February 3, 2022, 1:56pm

Finally refresh schedule work correctly!
Thank you very very much for your support!!