1

I’m looking into deploying a small redash instance into AWS using ECS. Are there any recommendations on updates to the docker-compose files (eg mem_limit for each of the services), and recommended # of instances/configuration of ECS using the console etc.

Any recommendations, things to look out for, considerations that need to be made when going this route would be greatly appreciated. I’m looking to keep costs down as much as possible on the AWS side. Is ECS the way to go? Or multiple EC2 instances?

A little about the intended use. It would probably be used by no more than 5 concurrent users at a time, OKTA integration, and would have no more than 25 “active” users. There may be a few customizations to the codebase but they would likely be cosmetic, and maybe the odd pull-request merge until things are merged into the main branch.

Thanks!

2

Anyone doing this successfully?

3

I never tried using the docker-compose files directly on ECS. Would love to hear if it worked for you. I’m running app.redash.io on ECS. I do have occasional issues, but I still don’t know if they are related to ECS and anyway related to high load which I doubt you will have.

Regardless here’s a few comments:

  • I wouldn’t run Redis or PostgreSQL on ECS for Redash. Use RDS and ElastiCache instead. If you really want to keep costs down, maybe run Redis on ECS but definitely not Postgres.
  • Memory limits depends on usage, I would probably give at least 450MB for each Redash service.
  • At the minimum split the services into: server, scheduler, worker. Once usage grows, split worker into scheduled queries worker and adhoc queries worker (by adjusting the QUEUE environment variable).
  • As usage grows, you will probably need to adjust the number of workers (mainly for the workers, but maybe for the server too).

You probably have more questions, so please ask.

4

Hi
I used the docker-compose production file from git and added redash to ecs and currently the query gets stuck in queue and it is not getting scheduled. No errors in debug logs. Do you know what might be the possible reason for this ?.

redis is in elasticache and postgres in RDS. the database was created by creating it locally and importing it to rds.

5

The https://github.com/getredash/redash/blob/master/docker-compose.production.yml seems to be wrong. Once i added scheduler, server, worker and nginx it started working.

6

Hi,

FYI, I have made a Cloudformation template to host Redash v8.0 on ECS. You could launch it within a few minutes.


Hope you can find it useful and give some feedback.

3 Likes
7

Wow! You just saved me a TON of time. Thank you so much. Send me your venmo and ill buy you a beer. (or in person if you are bay area)

8

Haha, thank you for generosity. Unfortunately I am not in Bay area. Just star the repository and share it someone else who might need it.

2 Likes
9

Hello,

First of all, thank you for this useful utility.

I am done till step 4 and for step 5, I am not able to understand which file to give as cloudformation template? I tried giving the s3 bucket path as you mentioned, but it is not correct as I am getting error.

i am looking help on below point

Create a Cloudformation from AWS Web console with the name you have decided
Upload files under a S3 bucket. And put the URL of S3 bucket in TemplateBaseURL parameter of the stack.

10

@arikfr - you say you run them as separate services. Is there a need for the different ReDash services to intercommunicate, or is all communication done via Redis?
Is there an architecture diagram/document of the desired setup or examples using terraform?

Thank you!
I am trying to get a POC up and running to demo internally.

11

This is correct: only nginx communicates with the web server (which can be a single ECS service with two tasks). Other than that all communication happens via Redis and/or the database.

2 Likes