So the queue wasn't running. Mail did come in now.
You can keep blomstra/flarum-redis installed, but you still have to set up either the worker daemon or the cronjob. You can use the same instructions for the cronjob as for the database queue driver:
sudo crontab -u www-data -e
and then on one row insert:
* * * * * cd /path/to/flarum && php flarum queue:work --stop-when-empty