I think I need to build a custom query runner for my use case. I am using MySQL and the existing runner works as intended, but the way things are sharded in my system I need to execute the query against dozens of identical-schema DBs spread out across multiple servers, and then combine the results at the end.
My plan is to build a custom runner to achieve this. However, I notice that there does not appear to be a way in the existing framework for a runner to push status updates to the UI while the query is in progress. Since my queries typically take several minutes to complete, and occasionally will time out or fail for one of the many DBs I execute against, I would like to be able to provide UI updates during the process so that the user can understand what % of the way it is completed and what specific shard it is querying at any given point. Additionally, at the end I would like to provide a summary such as “Queried 89/92 shards successfully; shards f7, d19, and c49 timed out.”