Missing JSON data source

#1

Issue Summary

Hi all, sorry if this is a dumb question. I’ve configured an hosted redash instance but I cannot find the JSON data source from the list of available ones. Am I missing something? Is there a way to troubleshoot this?
I think I have installed all the requirements, following is the list of all the data sources available:

Technical details:

  • Redash Version: git master checked out on 2019-03-20
  • Browser/OS: Ubuntu 18.04
  • How did you install Redash: with the legacy bootstrap.sh script
0 Likes

#2

Did you try the “Url”_Datasource? (first in the last row on your screenshot) :slight_smile:

0 Likes

#3

The JSON data source hasn’t been deployed in the open source version yet.

0 Likes

#4

Any idea when this data source will be released for open source?

1 Like

#5

Ah, I see… thanks for pointing this out susodapop. I hope it will be released soon :slight_smile:

1 Like

#6

This would be great, can’t wait for this too :smiley:

0 Likes

#7

@ricpersi It’s definitely possible to use JSON data sources in the Open Source (self hosted) version already, as long as the JSON is formatted how Redash wants it (there’s a doc).

Setting it up is a bit non-obvious, but is doable too.

I can point you to a working online data source, and take some screenshots of the setup, if that’d help?

0 Likes

#8

I hope this has been alleviated. We recently updated the docs for the URL data source to make this as clear as possible.

0 Likes

#9

The JSON docs piece looks ok, but the URL setup on the same page isn’t clear (my opinion).

With the URL setup, there are three parts needed for anything useful:

  1. The URL setup data source needs creating
  2. A “Query Results” data source needs creating
  3. The desired query then needs creating, to extract the data and present it usefully.

The URL setup doc only covers step 1, and just points to the doc about step 2.

For people already familiar with steps 2 and 3, that’s probably fine. For new users, unfamiliar with any of the steps, it’s not what they really need. :wink:

I’ll be documenting this stuff anyway from the DBHub.io point of view over the next week or so (I’m finally up to doing our docs), and our doc content is all OSS, so you’re welcome to copy/steal/etc the doc bits we do if that’ll help. :slight_smile:

2 Likes

#10

This is really good feedback. I will spend some time on this to update our docs sufficiently. Looking forward to seeing what you put out as well :slight_smile:

0 Likes

#11

Hi! That would definitely help. I will try having a deeper look at the docs but please let me know if you manage to take some screenshots to better understand how to configure the url datasource… thanks!

I just had a quick look at the URL datasource documentation. Unfortunately I don’t have control over the API I’m trying to query, so there’s no way I can make it output an object with columns and rows as it would be expected. Seems like I will have to wait for the JSON datasource.

Another option would be (but this is a bit off topic) to have some kind of webservice that sits in the middle, so that I can query it from redash. This webservice would then query the original API and transform the output so that it can be read by redash. Do you know if something like this already exists?

0 Likes

#12

I’m also using the URL DS at present, but would be very keen to shift to the JSON version when it’s available in the github codebase. Is there an ETA?

0 Likes

#13

No ETA yet :-\ The main blocker is that I’m hoping to make the JSON a backward compatible replacement for the current URL data source, so we don’t have two of them. I might give up on this and just release as is.

0 Likes

#14

Yep, there’s nothing stopping the release of an UrlJSON™ :wink: data source later on, if a good way to combine them is figured out.

1 Like

#15

@justinclift Suddenly the omission makes sense - with the hosted service when UrlJSON is ready some hacky scripts, with some manual checks, can be written/run to migrate existing users over to the eventual combined DS. But there’s no desire to spend enough time on those scripts to make them robust enough to be included in the OSS codebase - so just avoid having that problem to solve in the first place.

That said, @arikfr perhaps release them into a branch and you might find you get some help from the community in making the new JSON DS backwards compatible with the old URL DS. :slight_smile:

0 Likes

#16

Fair point. Added to my todo to create a draft Pull Request for this.

0 Likes

#17

Looking at the docs for the JSON data source… without knowing the details, it already looks like it’d be compatible with URL data sources. eg it can be given a URL to pull data from, and has selectors to pick desired fields out of it

What’s the bit that’s not working? :slight_smile:

0 Likes

#18

Yes, it is a superset of the URL functionality. But I want to be able to run existing URL queries with the new data source. This means that we either need to support the format of having just a URL in the query or run a migration when introducing this data source.

0 Likes

#19

Does anyone have a few minutes to look through these initial “How to set up a URL data source” docs, that I’ve been creating today?

https://docs.dbhub.io

Mainly looking for feedback on whether the steps as shown are really the correct way to do things. :wink:

The source for the docs are here if that’s of interest to anyone:

Experimenting with system called “Bookdown”, that can generate output in HTML, PDF, ePub, and a few other formats.

Not really sure I like it so far. May just revert to using Hugo (static website generator) as the output from that seems cleaner, though it’s HTML only. To be decided… :wink:

0 Likes

#20

Looks good! Only thing I would change is to not bother with creating a URL data source for each URL, but rather create a single data source without specifying a URL base path and then pass the URL in the query text.

1 Like