Feature idea -- github integration for sql version management

Hi there – just a few days ago, we instantiated and deployed redash (self-hosted) to our 150 person company. It’s enormously helpful. Self-service is real, and people are sending each other URLs to queries and data sets. Jumping through IT/Infra hoops to get a SQL account and VPN and ssh tunnels are now things of the past.

We have a new problem with knowledge management. The data science team has a git repo where we have standardized and version controlled SQL. Now, we have a proliferation of increasingly entropic SQL inside redash. The query objects are mushrooming and will become impossible to curate.

It would be amazing if we had a way to link to our git instance (we are GH enterprise users). If we can vacuum in our SQL repo, even if read-only, with a trace URL of where it comes from, that will make things better. Future – make it so a PR gets optionally generated if the original SQL is modified (in redash there are 2 uses, a scratch throwaway space and an “official” scenario where we want to version control SQL).

We don’t need Looker-style “LookML” if can interface with Git.

Thank you and keep up the good work.

4 Likes

This is great :smiley:

You mean create Redash queries based on those SQL files in Git?

I assume you mean if it’s modified in Redash?

– Yes! We have tons of version controlled SQL definitions in git (just strings defined in python, sometimes parametrized with curlies)

– Yes! Sometimes I can see that people will run the SQL unmodified. Often they will tweak something. If they change it, sometimes they may want to change what is in git. This is a secondary request, less important than above which is reading the source of truth from git.

I was hoping to find the time to follow up with more details. But unfortunately this didn’t happen, so figured I will share some initial thoughts in case it might be helpful:

  • We have an example script that syncs queries to files, but you can do the same in the opposite direction (sync the files to Redash queries).
  • You can update Redash’s Query Snippets from GitHub using the API. I’m not sure if it’s useful enough, because currently snippets are staticly inserted into the query. So existing queries that used a snippet won’t get updated.
1 Like