Docker issue with disk being full

Issue Summary

I tried accessing my hosted Redash instance the other day and got an Internal Server Error. When I tried running Docker commands it gives me: [23955] INTERNAL ERROR: cannot create temporary directory! Upon Google search, it was because my disk was full and I solved the issue by running docker system prune --all. My questions:

  1. Is there a way of preventing this in the future? This has actually happened the 2nd time now and both times the downtime caught me by surprise.
  2. Does this happen to anyone else? What is it about running Redash on Docker causes it to use so much disk space? Can I fix this by allocating more disk space (if so how?) Is there some sort of on going maintenance I should periodically to prevent this from happening again? Or is there some sort of leakage in the app that’s causing it use more disk space than expected? Is it because the database is fill up with query results? Or because logs are piling up?

Here’s what my syste looks like:
root@ip-xx-xxxx-xxx:/opt/redash# df -h
Filesystem Size Used Avail Use% Mounted on
udev 2.0G 0 2.0G 0% /dev
tmpfs 394M 41M 353M 11% /run
/dev/xvda1 7.7G 7.7G 0 100% /
tmpfs 2.0G 0 2.0G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup
/dev/loop1 56M 56M 0 100% /snap/core18/1988
/dev/loop2 34M 34M 0 100% /snap/amazon-ssm-agent/3552
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/85eadaa33cb6ae3b127b8118c7ab67d576f4a143f2e1369d335884bd11a79a13/merged
shm 64M 0 64M 0% /var/lib/docker/containers/939d9685cc9e6019339b101b2021992c08e590cfb99af537bbda4e0edb98edc5/mounts/shm
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/41fe649e21d0478e68d58b03a31d1fdb270f3f4711ad15dccb68dd1999a00246/merged
shm 64M 4.0K 64M 1% /var/lib/docker/containers/6209fdd3e9b0714cebe6fafbcbac425c909b9d94510a5732132198af111dd983/mounts/shm
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/6caac15d05f118538abcbbe878d72908f3e2fa8654395baa0613d479aa3692af/merged
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/4b38d76ec0230cac2a142d49267e432c31a71a0515e62e522edccef4e7ed4590/merged
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/0c4d1c9837538e2e4037690bb12c8bce3afc6c1dfffd1892b18378196ccd3840/merged
shm 64M 0 64M 0% /var/lib/docker/containers/f3e99ddc7dee98359c49868359abe3e3171f567e2f196d902677abecf8090856/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/836d54eb370c78a663c29fcf38fb7d0cf26e5640761594504a0613b372860b71/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/357ebc921198b449dffd79086db85be851435d30fa37c2aae3ee329d664df8e9/mounts/shm
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/0f4909c452cec127de4f3b58ba4c6e710204c7ba491f733121846fcb17b19024/merged
overlay 7.7G 7.7G 0 100% /var/lib/docker/overlay2/91277a03c8aecfd40f5ef0e5622ca878fecd9e0b63efa2fa49910321b07b01c5/merged
shm 64M 0 64M 0% /var/lib/docker/containers/e8c3c1c2b54e4d644c0b0f5248fc1ec62ae10f18382daa2daf89bd7672db72e6/mounts/shm
shm 64M 0 64M 0% /var/lib/docker/containers/dbe145332eb44765dd534a03d564534f7159079839c9063e2e9c77a10e292e23/mounts/shm
/dev/loop0 56M 56M 0 100% /snap/core18/1997
/dev/loop5 33M 33M 0 100% /snap/snapd/11588
/dev/loop3 33M 33M 0 100% /snap/snapd/11841
tmpfs 394M 0 394M 0% /run/user/1000

Technical details:

  • Redash Version: 9
  • Browser/OS: Mac
  • How did you install Redash: AWS AMI