Upgrade from 4.0.1 to redash 7.0.0 giving "Internal Server Error"

tanay · September 18, 2019, 10:39am

Receiving following error under /var/log/supervisor/redash_server-stderr—supervisor-xxxxxxx.log

[2019-09-18 08:39:50,528][PID:150][ERROR][redash] Exception on / [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask_restful/__init__.py", line 271, in error_router
    return original_handler(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python2.7/dist-packages/flask_login/utils.py", line 228, in decorated_view
    return func(*args, **kwargs)
  File "/opt/redash/redash-7.0.0/redash/handlers/static.py", line 27, in index
    return render_index()
  File "/opt/redash/redash-7.0.0/redash/handlers/static.py", line 18, in render_index
    response = send_file(full_path, **dict(cache_timeout=0, conditional=True))
  File "/usr/local/lib/python2.7/dist-packages/flask/helpers.py", line 536, in send_file
    file = open(filename, 'rb')
IOError: [Errno 2] No such file or directory: '/opt/redash/redash-7.0.0/redash/settings/../../client/dist/index.html'

What could be the possible issue ?.. I’m using legacy deployment (not using docker-compose approach, modified bootstrap.sh file)

jesse · September 18, 2019, 12:28pm

It looks like you haven’t built the front-end. Try running npm run build.

tanay · September 19, 2019, 8:37am

Receiving following error now, after running “npm run build”

> redash-client@7.0.0 build /opt/redash/redash-7.0.0
> npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack
> redash-client@7.0.0 clean /opt/redash/redash-7.0.0
> rm -rf ./client/dist/

e[91msh: 1: node: not found
e[0me[91m
e[0me[91mnpme[0me[91m e[0me[91mERR! Linux 4.14.33-51.37.amzn1.x86_64
e[0me[91mnpm ERR! argve[0me[91m "/usr/bin/nodejs" "/usr/bin/npm" "run" "build"
npm ERR! node v4.2.6
npm e[0me[91mERR!e[0me[91m npm  v3.5.2
npm e[0me[91mERR! filee[0me[91m sh
e[0me[91mnpm ERR! code ELIFECYCLE
npm ERR! e[0me[91merrno ENOENT
npm ERR! syscall spawn
npm ERR!e[0me[91m redash-client@7.0.0 build: `npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack`
e[0me[91mnpm ERR! spawn ENOENT
npm ERR!e[0me[91m 
npm ERR! Failed at the redash-client@7.0.0 build script 'npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack'.
npm ERR!e[0me[91m Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the redash-client package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack
e[0me[91mnpm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs redash-client
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls redash-client
e[0me[91mnpm ERR!e[0me[91m There is likely additional logging output above.
e[0me[91m
e[0me[91mnpm ERR! Please include the following file with any support request:
npm ERR!e[0me[91m     /opt/redash/redash-7.0.0/npm-debug.log
e[0mThe command '/bin/sh -c cd /opt/redash/redash-7.0.0/ && npm run build' returned a non-zero code: 1

jesse · September 19, 2019, 10:49am

Next, upgrade Node from v4 to at least v10. Your system version of Node is pretty old.

tanay · September 19, 2019, 12:08pm

Hey @jesse, while upgrading node v4 to v12, receiving following error

redash-client@7.0.0 build /opt/redash/redash-7.0.0
npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack

redash-client@7.0.0 clean /opt/redash/redash-7.0.0
rm -rf ./client/dist/

e[91minternal/modules/cjs/loader.js:775
throw err;
^

Error: Cannot find module ‘/opt/redash/redash-7.0.0/node_modules/.bin/webpack’
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:772:15)
at Function.Module._load (internal/modules/cjs/loader.js:677:27)
at Function.Module.runMain (internal/modules/cjs/loader.js:999:10)
at internal/main/run_main_module.js:17:11 {
code: ‘MODULE_NOT_FOUND’,
requireStack:
}
e[0me[91mnpme[0me[91m ERR! code ELIFECYCLE
e[0me[91mnpm ERR! errno 1
e[0me[91mnpme[0me[91m ERR! redash-client@7.0.0 build: npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack
npm ERR!e[0me[91m Exit status 1
e[0me[91mnpm ERR!e[0me[91m
npm ERR!e[0me[91m Failed at the redash-client@7.0.0 build script.
npme[0me[91m ERR! This is probably not a problem with npm. There is likely additional logging output above.
e[0me[91mnpm WARNe[0me[91m Local package.json exists, but node_modules missing, did you mean to install?
e[0me[91m
e[0me[91mnpm ERR! A complete log of this run can be found in:
npm ERR!e[0me[91m /root/.npm/_logs/2019-09-19T12_04_47_988Z-debug.log
e[0mThe command ‘/bin/sh -c cd /opt/redash/redash-7.0.0/ && npm run build’ returned a non-zero code: 1

I’m using following command to install nodejs

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
apt-get install -y nodejs

What could be the possible issue ?

jesse · September 19, 2019, 12:13pm

Look closely at the stack trace. It says that build failed because you’re missing the webpack module. Run npm install before you try to build again.

tanay · September 20, 2019, 6:40am

Hi @jesse, After “npm install” seeing folllowing error. I’m not a nodejs expert. So it is hard to figure out the issue.

e[91mnpme[0me[91m WARN deprecated bfj-node4@5.3.1: Switch to the bfj package for fixes and new features!
e[0me[91mnpm WARN deprecated browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
e[0me[91mnpm WARN e[0me[91mdeprecated left-pad@1.3.0: use String.prototype.padStart()
e[0me[91mnpm WARN deprecated @mapbox/gl-matrix@0.0.1: This
e[0me[91mnpm WARN e[0me[91mdeprecated flatten@1.0.2: I wrote this module a very long time ago; you should use something else.
e[0me[91mnpm WARN deprecated core-js@1.2.7: core-js@<2.6.8 is no longer maintained. Please, upgrade to core-js@3 or at least to actual version of core-js@2.
e[0me[91mnpm WARN e[0me[91mdeprecated os-homedir@2.0.0: This is not needed anymore. Use require('os').homedir() instead.
e[0me[91mnpm ERR! e[0me[91mcodee[0me[91m ENOENT
npm ERR! syscalle[0me[91m spawn git
e[0me[91mnpm ERR! path git
npm ERR! errnoe[0me[91m ENOENT
e[0me[91mnpm ERR! enoente[0me[91m Error while executing:
npm ERR! enoent undefined ls-remote -h -t https://github.com/restorando/cornelius.git
npm ERR! enoent
npm ERR! e[0me[91menoent
npm ERR!e[0me[91m enoent spawn git ENOENT
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent
e[0me[91m
e[0me[91mnpme[0me[91m ERR!e[0me[91m A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-09-20T08_42_53_599Z-debug.log
e[0mThe command ‘/bin/sh -c npm install’ returned a non-zero code: 1

Can you please let me know what could be the possible issue ?

tanay · September 20, 2019, 1:09pm

Hey @jesse, I’ve modified few steps before I performed npm install. Now npm install step is passing. But, during npm run build getting following error

redash-client@7.0.0 build /opt/redash/redash-7.0.0
npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack

redash-client@7.0.0 clean /opt/redash/redash-7.0.0
rm -rf ./client/dist/

e[91m
WARNING: We noticed you’re using the useBuiltIns option without declaring a core-js version. Currently, we assume version 2.x when no version is passed. Since this default version will likely change in future versions of Babel, we recommend explicitly setting the core-js version you are using via the corejs option.

You should also be sure that the version you pass to the corejs option matches the version specified in your package.json's dependencies section. If it doesn’t, you need to run one of the following commands:

npm install --save core-js@2 npm install --save core-js@3
yarn add core-js@2 yarn add core-js@3

e[0mHash: ea3e183b6921d81f9b09
Version: webpack 4.40.2
Time: 125161ms
Built at: 09/20/2019 1:06:07 PM
102 assets
Entrypoint app = vendors~app.41e2ceb2d218f2baf6b3.css vendors~app.41e2ceb2d218f2baf6b3.js vendors~app.41e2ceb2d218f2baf6b3.css.map vendors~app.41e2ceb2d218f2baf6b3.js.map app.449cf4d9a133b9678c6d.css app.449cf4d9a133b9678c6d.js app.449cf4d9a133b9678c6d.css.map app.449cf4d9a133b9678c6d.js.map
Entrypoint server = server.7d948fd1146c89c80d5b.css server.7d948fd1146c89c80d5b.js server.7d948fd1146c89c80d5b.css.map server.7d948fd1146c89c80d5b.js.map

WARNING in ./client/app/components/PreviewCard.jsx
Module Warning (from ./node_modules/eslint-loader/index.js):

/opt/redash/redash-7.0.0/client/app/components/PreviewCard.jsx
36:53 warning ‘user.id’ is missing in props validation react/prop-types
63:66 warning ‘dataSource.id’ is missing in props validation react/prop-types

2 problems (0 errors, 2 warnings)

@ ./client/app/pages/users/UsersList.jsx 38:0-59 138:33-48
@ ./client/app/pages sync ^((?![\/.]test[\./]).)*.jsx?$
@ ./client/app/config/index.js
@ ./client/app/index.js
@ multi ./client/app/index.js ./client/app/assets/less/main.less ./client/app/assets/less/ant.less

WARNING in ./client/app/components/dashboards/EditParameterMappingsDialog.jsx
Module Warning (from ./node_modules/eslint-loader/index.js):

/opt/redash/redash-7.0.0/client/app/components/dashboards/EditParameterMappingsDialog.jsx
87:16 warning ‘toastr.error’ is missing in props validation react/prop-types

1 problem (0 errors, 1 warning)

@ [\/.]test[\./]).)*.jsx? (./client/app/components sync ^((?![\\\/.]test[\\.\/]).)*\.jsx?) ./dashboards/EditParameterMappingsDialog.jsx
@ ./client/app/config/index.js
@ ./client/app/index.js
@ multi ./client/app/index.js ./client/app/assets/less/main.less ./client/app/assets/less/ant.less

WARNING in ./client/app/components/groups/GroupName.jsx
Module Warning (from ./node_modules/eslint-loader/index.js):

/opt/redash/redash-7.0.0/client/app/components/groups/GroupName.jsx
17:49 warning ‘group.type’ is missing in props validation react/prop-types

1 problem (0 errors, 1 warning)

@ ./client/app/pages/groups/GroupDataSources.jsx 49:0-54 257:29-38
@ ./client/app/pages sync ^((?![\/.]test[\./]).)*.jsx?$
@ ./client/app/config/index.js
@ ./client/app/index.js
@ multi ./client/app/index.js ./client/app/assets/less/main.less ./client/app/assets/less/ant.less

ERROR in ./client/app/components/admin/CeleryStatus.jsx
Module Error (from ./node_modules/eslint-loader/index.js):

/opt/redash/redash-7.0.0/client/app/components/admin/CeleryStatus.jsx
3:1 error @/services/ng import should occur after import of lodash/values import/order

1 problem (1 error, 0 warnings)
1 error and 0 warnings potentially fixable with the --fix option.

@ ./client/app/pages/admin/tasks/index.js 3:0-59 11:56-68
@ ./client/app/pages sync ^((?![\/.]test[\./]).)*.jsx?$
@ ./client/app/config/index.js
@ ./client/app/index.js
@ multi ./client/app/index.js ./client/app/assets/less/main.less ./client/app/assets/less/ant.less
Child html-webpack-plugin for “index.html”:
1 asset
Entrypoint undefined = index.html
Child html-webpack-plugin for “multi_org.html”:
1 asset
Entrypoint undefined = multi_org.html
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!client/app/components/QueryEditor.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!client/app/components/app-header/app-header.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!client/app/components/queries/ScheduleDialog.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!client/app/pages/dashboards/dashboard-list.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!client/app/pages/queries-list/queries-list.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/cornelius/src/cornelius.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/gridstack/dist/gridstack.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/leaflet-fullscreen/dist/leaflet.fullscreen.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/leaflet.markercluster/dist/MarkerCluster.Default.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/leaflet.markercluster/dist/MarkerCluster.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/leaflet/dist/leaflet.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–6-1!node_modules/pivottable/dist/pivot.css:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/assets/less/ant.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/assets/less/main.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/assets/less/server.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/HelpTrigger.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/ParameterMappingInput.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dashboards/AddTextboxDialog.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dashboards/gridstack/gridstack.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dashboards/widget-dialog.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dashboards/widget.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dynamic-table/dynamic-table.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/dynamic-table/json-cell/json-view-interactive.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/empty-state/empty-state.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/components/layouts/content-with-sidebar.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/pages/dashboards/dashboard.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/pages/users/settings.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/redash-font/style.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/visualizations/funnel/funnel.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/visualizations/pivot/pivot.less:
Entrypoint mini-css-extract-plugin = *
Child mini-css-extract-plugin node_modules/css-loader/index.js??ref–7-1!node_modules/less-loader/dist/cjs.js??ref–7-2!client/app/visualizations/table/table-editor.less:
Entrypoint mini-css-extract-plugin = *
e[91mnpm ERR!e[0me[91m code ELIFECYCLE
e[0me[91mnpm e[0me[91mERR! errnoe[0me[91m 2
e[0me[91mnpme[0me[91m ERR! redash-client@7.0.0 build: npm run clean && NODE_ENV=production node --max-old-space-size=4096 node_modules/.bin/webpack
npm e[0me[91mERR! Exit status 2
e[0me[91mnpm ERR!
e[0me[91mnpm ERR! Failed at the redash-client@7.0.0 build script.
npme[0me[91m ERR! This is probably not a problem with npm. There is likely additional logging output above.
e[0me[91m
npme[0me[91m ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-09-20T13_06_08_106Z-debug.log
e[0mThe command ‘/bin/sh -c npm run build’ returned a non-zero code: 2

jesse · September 20, 2019, 1:47pm

What steps made npm install work? The issue two posts ago was that you installed node v10+ but you were still using node v4 to run the install script. The errors in your newest post still suggest a version mismatch and some weird flags. I don’t believe we run the linter in production.

jesse · September 20, 2019, 1:51pm

Thinking about this further, I think something went haywire during your upgrade. Many of those errors are because of outdated NPM packages. What steps did you take to upgrade your installation?

tanay · September 20, 2019, 2:17pm

Hi @jesse,

Steps I followed to install node 12 version

curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
apt-get install -y nodejs

Under /opt/redash/redash-7.0.0/

rm -rf package-lock.json node_modules/
npm cache clear --force
npm install
npm run build

tanay · September 24, 2019, 10:52am

@jesse Anything else should I try apart from the steps I mentioned above ?

jesse · September 24, 2019, 10:54am

Can you tell me about your installation? How did you set up Redash? Docker?

tanay · September 24, 2019, 10:57am

@jesse

Here is the Snippet of my installation. We modified .env, bootstrap.sh and supervisord.conf as per requirements.

FROM ubuntu:16.04

RUN apt-get update && apt-get install -y
curl
git
postgresql
redis-server
sudo
wget
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*

RUN adduser --system --no-create-home --disabled-login --gecos “” redash

#COPY .env /opt/redash/
#RUN chown redash /opt/redash/.env

COPY supervisord.conf /etc/supervisor/conf.d/

RUN service postgresql start && service redis-server start
COPY bootstrap.sh /opt/redash/

RUN /bin/sh /opt/redash/bootstrap.sh

COPY .env /opt/redash/redash-7.0.0/
RUN chown redash /opt/redash/redash-7.0.0/.env

#RUN service redis-server start

RUN service supervisor stop

RUN curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -

RUN apt-get install -y nodejs build-essential

WORKDIR /opt/redash/redash-7.0.0/

RUN rm -rf package-lock.json node_modules/
RUN npm cache clear --force
RUN npm update
RUN npm install --no-optional

RUN npm install --save core-js@^3

#create build manifest
RUN npm run build
…
…
…
…

jesse · September 24, 2019, 11:08am

I think you will have better success if you manually upgrade your V4 instance to V5, V6, and then V7.

tanay · September 26, 2019, 6:40am

@jesse

I successfully managed to upgrade Redash from 4.0.1 to 7.0.0. I think Work directory for Dockerfile to run npm install and npm run build command must be under /opt/redash/redash-7.0.0/client, NOT the/opt/redash/redash-7.0.0/

Thanks for your help!

▼Categories

▼Tags

Upgrade from 4.0.1 to redash 7.0.0 giving "Internal Server Error"