pdpinch
1
I have a pivot table visualization that is crashing my browser (in Firefox, I get the message “A web page is slowing down your browser”
The underlying query is large-ish for our uses (19564 rows) but it returns fine in a table visualization.
Why is this happening? Do I need to edit the visualization, or is it something else?
k4s1m
2
Hmmm what kind of aggregations does your pivot table do? That is a lot of data for the pivot table to process through since it all happens in the front-end of your browser. The normal table visualization is quicker because it doesn’t do any add/subtract/divide operations.
this is a known problem with Redash frontend. Anything over about 5-10K rows really slows down the browser.
in Firefox about:config you can change dom.max_script_run_time from 10->20 (seconds) to at least give the browser longer to throw the error.
you might also try upping javascript.options.mem.nursery.max_kb, but I can’t say that I’ve found that that helps a lot.
It’s unfortunate because 5-10K rows is something very easy for Excel to handle. Zero Excel is a great goal.
Firefox debugger output:
Just a followup here. It seems that it’s CPU bound not memory bound (which would explain why max_kb isn’t helping).
As @k4s1m said, the pivot table component performs the pivot operation in the web browser. Comparing Excel and a web browser isn’t really valid, web browsers are not designed to manipulate large amounts of data in memory.
If you need to pivot large amounts of data i.e. more than a few thousand rows, then I suggest performing the pivot in the SQL query i.e. on the data source, not in the web browser. If the requirement is to pivot a large amount of data and this can’t pushed down to the data source then potentially Redash is not the appropriate tool.
Due to the fact this is occurring when it’s a very small amount of data (<10Kb data) but large numbers of rows, and the browser is not displaying any issues with memory, I am positing this is a problem with the frontend software rather than a browser limitation.
In this scenario the browser is the front end software. As far as I understand it, for the pivot table component all pivot calculations are run directly in the browser, not on any of the Redash containers or on the data source itself. The pivot visualisation component is useful for working with small amounts of data and I’ve used it successfully when the query result was a few thousand rows, but after that performance drops off. I don’t think there’s a software fix for this, it’s just a limitation of what you can do in a browser.
Comparable tools such as Tableau, Quicksight and Microstrategy rely on this kind of work being done in memory on a server and only the result is rendered by the browser, which is not how Redash works.
The front end software is javascript and is handled by a Javascript engine. The fact that this only occurs while using certain pivot features suggests there could be a bug in the Redash javascript.
k4s1m
9
The pivot table comes from plotly. It’s called react-pivottable and is also open source. But it’s not developed by Redash directly. If you found a bug it would be best to report it to those developers. https://github.com/plotly/react-pivottable
I agree it sounds like a bug since 10kb shouldn’t be a problem.
