CPU Usage constantly rising in workers

Issue Summary

We have been experiencing constant and rising CPU usage from redash workers without being able to understand why.

We use the below configuration to run celery:
exec /usr/local/bin/celery worker --app=redash.worker -c30 -Q$QUEUES -linfo --max-tasks-per-child=100

REDASH_WEB_WORKERS=8

What we are seeing is the following:

CPU keeps rising until we need to restart the service.

Any ideas?

Thanks!

Lucas

Technical details:

  • Redash Version: 8.0.0.0-beta
  • Browser/OS: Chrome/Windows and OSX
  • How did you install Redash: Docker image running in kubernetes with external redis cache.

I just responded to a similar issue here so you can take a look: