Lukeman yeah it's a downside I have noticed.
I think the easiest solution for me to implement will be some sort of tag restriction.
I could also offer a group selection but unfortunately the way the Flarum permission system is designed doesn't allow querying for discussions a given group can see, we always need to provide a user. So another option would be to select one user and whatever that user can see is used as the value.
Not sure when I'll do that. Maybe during one of my future livestreams.