I’m hoping to add ability to disable/remove users in one of the next versions.
As for setting default password – while it’s better to have emails setup, it’s not really mandatory. When you create the user, the system shows you the invite link which you can pass on to the user yourself.
The best way to configure a mail server is to use a service like MailGun. They have a generous free tier of 10K/mo messages which should be more than enough for Redash.