Redash on AWS ECS


#1

I’m looking into deploying a small redash instance into AWS using ECS. Are there any recommendations on updates to the docker-compose files (eg mem_limit for each of the services), and recommended # of instances/configuration of ECS using the console etc.

Any recommendations, things to look out for, considerations that need to be made when going this route would be greatly appreciated. I’m looking to keep costs down as much as possible on the AWS side. Is ECS the way to go? Or multiple EC2 instances?

A little about the intended use. It would probably be used by no more than 5 concurrent users at a time, OKTA integration, and would have no more than 25 “active” users. There may be a few customizations to the codebase but they would likely be cosmetic, and maybe the odd pull-request merge until things are merged into the main branch.

Thanks!


#2

Anyone doing this successfully?


#3

I never tried using the docker-compose files directly on ECS. Would love to hear if it worked for you. I’m running app.redash.io on ECS. I do have occasional issues, but I still don’t know if they are related to ECS and anyway related to high load which I doubt you will have.

Regardless here’s a few comments:

  • I wouldn’t run Redis or PostgreSQL on ECS for Redash. Use RDS and ElastiCache instead. If you really want to keep costs down, maybe run Redis on ECS but definitely not Postgres.
  • Memory limits depends on usage, I would probably give at least 450MB for each Redash service.
  • At the minimum split the services into: server, scheduler, worker. Once usage grows, split worker into scheduled queries worker and adhoc queries worker (by adjusting the QUEUE environment variable).
  • As usage grows, you will probably need to adjust the number of workers (mainly for the workers, but maybe for the server too).

You probably have more questions, so please ask.


#4

Hi
I used the docker-compose production file from git and added redash to ecs and currently the query gets stuck in queue and it is not getting scheduled. No errors in debug logs. Do you know what might be the possible reason for this ?.

redis is in elasticache and postgres in RDS. the database was created by creating it locally and importing it to rds.


#5

The https://github.com/getredash/redash/blob/master/docker-compose.production.yml seems to be wrong. Once i added scheduler, server, worker and nginx it started working.