I often find that I ruin permanently a Query by typing the parameter incorrectly, or wanting to change the parameters. There does not appear to be any way to delete a parameter from a Query - you’re stuck with it forever.
My preferred behaviour would be that clicking the Parameter cog brings up the parameter dialog, and that dialog has a delete parameter button on it.