akshay59
1
I see that Redash is taking a lot of space because of Query Result. Every day the new files size is around 80GB, even though i changed the default REDASH_QUERY_RESULTS_CLEANUP_MAX_AGE & REDASH_QUERY_RESULTS_CLEANUP_COUNT to remove any unused data still my DB size is increasing day by day. Is there a way to skip storing data in Query Result or cleaning of this data.
k4s1m
2
That’s a lot of disk space! You can’t root out the cache completely because the latest query result is always cached. How much data do you pull in your queries? If you routinely query 80GB of data then the cache is going to be that large.
arikfr
3
By default, Postgres doesn’t return used space to the system but reuses it for future data. I believe you can run VACUUM FULL to return disk space, but you better check the docs.
Shitij
4
A bit off topic, I think you need to rethink how redash is being used. This seems more like a raw data dump happening. (Assuming people are not running queries on a massive scale). Maybe have a limit on the number of rows returned? (Will have to fork to make your own changes here)
Redash is generally meant for 20-25 MB of aggregated which can be visualized.
vish
5
the query results had 11GB when I cleaned it and vacuumed.
the /var/docker is 200 GB
what else is taking up space ?
I used du -h to find out the PostgreSQL docker container was logging out a monster log file.
My CONTAINER_ID-json.log file was over 825 GB.
sudo du -h /var/lib/docker/containers
sudo du -h /var/lib/docker/containers/CONTAINER_ID
sudo ls -la /var/lib/docker/containers/CONTAINER_ID
parasiil
7
reducing logfile size in docker containers How to reduce the size of the Docker container log file | sleeplessbeastie's notes
Interesting that Redash docker-compose.yml does not have log rotation built in
