I think that's the intended behavior. There's no minimize button on mobile but that's essentially what the cross does.
It might make sense to add some logic to detect whether the composer is empty and close it immediately in that case. But there's no way to do this in the current Flarum version, we'd need to add a new extension endpoint so extensions can tell the composer whether anything was typed in their custom fields.