Schema refresh failed - MongoDB datasource

YakobovLior · June 19, 2019, 12:48pm

Hello,
I’m running latest redash docker-compose based, and I’m having issues with MongoDB datasource.
We are using AWS DocumentDB service and until a few days ago I was able to run queries from redash against the DocumentDB, but now when I’m trying to query I cannot get the database collections and therefore the query fails.
Anyone has a solution for this? I have tried running redash in debug mode but didn’t find anything useful.
Thanks,
Lior

YakobovLior · June 30, 2019, 1:15pm

Hello,
Any help? does anyone familiar with this error?

Thanks,
Lior

arikfr · July 3, 2019, 7:34am

Considering it was working, what changed between now and then? What errors do you see in the logs?

YakobovLior · August 5, 2019, 1:02pm

Hey Arik,
Actually I’m not familiar with any change, as I described earlier we are using AWS managed service so maybe something has changed but is out of my sight.
From other checks I saw that if I start my own MongoDB there are system and admin schemas which are missing in AWS DocumentDB, could that be the issue? (I was able to query my local MongoDB but still cannot query the DocumentDB).

Thanks,
Lior

YakobovLior · August 20, 2019, 7:21am

Hey Arik,
Any comments about my last response?
The usage of Redash against the DocumentDB is vital for us - I would appriciate any help possible.
Thanks,
Lior

jesse · September 5, 2019, 6:29pm

Hi @YakobovLior, are you able to query the DocumentDB from a command line or other software? Can you share the specific errors that you see?

YakobovLior · November 10, 2019, 12:26pm

Hey @jesse,
AWS DocumentDB is with MongoDB compatibility, therefore I’m able to query it for exmaple with mongo cli, MongoDB Compass tool, PyMongo library using Python and so on.
When I try to configure the DocumentDB as a MongoDB data source, it does work and also the test returns success, but when I’m trying to query the data source, this is the error I’m getting:

From AWS DocumentDB documentation, it says that the database does not contain system and admin schemas: (Functional Differences: Amazon DocumentDB and MongoDB - Amazon DocumentDB)

Could that be the issue here?

Thanks,
Lior

nodesocket · August 2, 2020, 5:04am

Hello. I am also having this exact same problem. Is AWS DocumentDB not supported? It’s so close it appears, just can’t load the collections. Probably would be a minor patch to the Redash MongoDB driver. Happy to assist is anybody can point in the right direction.

Able to connect to the AWS DocumentDB from Node.js and also using MongoDB Compass as well.

k4s1m · August 3, 2020, 4:26pm

Here’s the code Redash uses to get the schema for mongodb. I don’t use DocumentDB but maybe you can see what’s missing or broken here?

github.com

getredash/redash/blob/ecb9adf903842694ab01479727709064296ea029/redash/query_runner/mongodb.py#L229-L241


def get_schema(self, get_stats=False):
    schema = {}
    db = self._get_db()
    for collection_name in db.collection_names():
        if collection_name.startswith("system."):
            continue
        columns = self._get_collection_fields(db, collection_name)
        schema[collection_name] = {
            "name": collection_name,
            "columns": sorted(columns),
        }

    return list(schema.values())

nodesocket · August 4, 2020, 9:34pm

@k4s1m thanks for this. I am not a Python developer, I use Node.js. If I provision a test DocumentDB instance and give you credentials, do you think you could test the above code out against DocumentDB and find where it’s breaking? I just ask, because I don’t have any of the Python code setup or an environment.

nodesocket · August 4, 2020, 9:49pm

I also created an issue in Github for this.

▼Categories

▼Tags

Schema refresh failed - MongoDB datasource