Tools for aggregation / continuous queries


#1

Hello,

I have a newbie question: it’s more about the best tools to use alongside redash rather than redash itself. Let me give an example of what I’m looking for.

One of the visualization examples shows a count of queries over a 4 year period. I want to build graphs like this.

I could have a redash query like

select day,count(*) from queries group by day order by day

and schedule it to run every 24 hours. But of course each time the query runs, it will be processing 4 years of data.

So I would like to count the queries over the last 24 hours, and add that to a new timeseries. As far as I can see, that’s not something redash itself offers.

Does anyone have any suggestions for tools to manage and automate such scheduled queries?

Looking around, I’ve found that influxdb has continuous queries. Are there other platforms I should be looking at?

A fully streaming solution which updates the counts in real time as individual events come in - like pipelinedb or KSQL - is probably overkill here.

Thanks, Brian.