1

Hello,

I’m currently using Redash Hosted (SaaS), the version it shows is 7b026a19
Due to the service being taken down, I’d like to migrate to a self hosted solution.
I’ve tried spinning up an EC2 with the AMI that’s available, but it’s for version 8 of Redash, I’ve tried running the ‘redash-migration’ script and while some things copied successfully, some other failed.

What is the best course of action in this case?
Is it possible to get a DB dump of the Redash SaaS environment?

2

Hi there, redash-migrate is only designed to migrate to OSS Redash V10. It might work for V8 but we haven’t tested that since V8 is not longer supported.

Happy to help with the migration process if you have questions. You’ll want to recreate your EC2 instance and upgrade it to V10 first, though.

3

Would it be able to migrate the V7 SaaS to V10 (That I host)?

4

Yes. The instructions for redash-migrate state specifically it is built for that EXACT case :slight_smile:

5

Started with a new EC2 instance and upgraded Redash to v10.
When getting to the part of migrating the visualizations I get the following (this is not the first run, but same issue):

Importing visualizations...
Query 140614 - SKIP - All visualisations already imported
...
Query 207138 - SKIP - All visualisations already imported
list index out of range
Saving meta...
6

Thanks for reporting this. I’m pushing an update to redash-toolbelt shortly so that we can get a better traceback when this fails. Show full traceback when an exception is raised by susodapop · Pull Request #108 · getredash/redash-toolbelt · GitHub

We’ll release version 0.1.8 shortly and then you can re-run.

1 Like
7

Tried it with version 0.1.8 of the script, still get the same error and output.
I found this issue for now:

This line fails for queries that the only visualization type they have is not of type “TABLE”.

I’m not 100% if this sorting part is necessary or not, so I don’t know if it’s safe to try run without it. @jesse any insights?

8

Thanks for running the latest version! That’s massively helpful.

Okay so the reason for the sorting is to figure out which table is the default table for the query. The default table is created automatically when the query is created. We just added this logic in 0.1.7. It’s important because customisations to the default table at the origin were not duplicated at the destination without it.

Since you’re running the dev build, can you add a print statement to see what is the content of query["visualizations"] right about line 675?

9

Version 0.1.8 is now live on pypi so it can be installed with pip install --upgrade redash-toolbelt

1 Like
10

@jesse This is the value of query[“visualizations”]:

[
  {
    "id": 354489,
    "type": "CHART",
    "name": "",
    "description": "",
    "options": {
      "showDataLabels": true,
      "error_y": {
        "visible": true,
        "type": "data"
      },
      "numberFormat": "0,0[.]00000",
      "minColumns": 1,
      "defaultColumns": 3,
      "series": {
        "stacking": null,
        "error_y": {
          "visible": true,
          "type": "data"
        }
      },
      "globalSeriesType": "pie",
      "yAxis": [
        {
          "type": "linear",
          "title": {
            "text": "some text"
          }
        },
        {
          "title": {
            "text": ""
          },
          "type": "linear",
          "opposite": true
        }
      ],
      "minRows": 5,
      "percentFormat": "0[.]00%",
      "sortX": true,
      "seriesOptions": {
        "payments": {
          "zIndex": 0,
          "index": 0,
          "type": "pie",
          "yAxis": 0
        }
      },
      "valuesOptions": {
        "credit": {
          "color": "#799CFF"
        },
        "ach": {
          "color": "#604FE9"
        },
        "debit": {}
      },
      "xAxis": {
        "labels": {
          "enabled": true
        },
        "type": "-",
        "title": {
          "text": ""
        }
      },
      "dateTimeFormat": "YYYY-MM-DD HH:mm",
      "columnMapping": {
        "paymentMethod": "x",
        "payments": "y"
      },
      "textFormat": "",
      "defaultRows": 8,
      "customCode": "// Available variables are x, ys, element, and Plotly\n// Type console.log(x, ys); for more info about x and ys\n// To plot your graph call Plotly.plot(element, ...)\n// Plotly examples and docs: https://plot.ly/javascript/",
      "legend": {
        "enabled": true
      }
    },
    "updated_at": "2019-05-26T19:56:21.105Z",
    "created_at": "2019-05-26T19:51:41.801Z"
  }
]
11

Well there’s the problem. This query doesn’t have a table visualization :thinking:

Just to confirm: you haven’t manually deleted any visualizations at the database-level? If not, then I think we have a race-condition in our script or something like it. The script is expecting that there will always be at least one table visualisation. But in this case there isn’t one…

12

Personally I haven’t deleted anything myself, but I don’t know if it might have been done before by someone else.
Is there any workaround for this?

13

I need to poke through the code to see if I can reproduce this. I would recommend wiping your destination instance and trying again though…

14

I’ve ended up just adding a table to that query on the origin server and running it again, it worked.

Another problem I’ve encountered was that some queries had parameters that referred to other queries but these queries were archived in the origin server and thus not copied over to the destination server.
This caused an internal server error as the ID that was send was ‘None’ (because it couldn’t find the relevant query in the meta file).
We just removed these queries and continued, but this might be worth to add validation in code for this.

15

Unfortunately there’s no much we can do about the archived query issue. Archived queries are never sent through the API redash-migrate uses.

I’m glad this worked. I expected it would. But I wonder if there’s a bug in our code that somehow didn’t transfer the table. If so, then other users will encounter this same issue and may not be as technically capable as you to fix it.

1 Like