ProgWeb from PHP so that I save some performances on the server side
The number of comments isn't computed during rendering, it's cached in the comment_count column of the database and only updated when a post is created or deleted. There will be no performance difference having it public or hidden.
That being said, if you wanted to hide the value completely including from the API for privacy reason, you could use the same logic as this extension https://github.com/clarkwinkelmann/flarum-ext-see-past-first-post/blob/2306330dc8e946109ae438a7cec1a19f3a55ca01/src/Extenders/DiscussionAttributes.php#L33-L37
Doing that alone won't prevent someone from manually requesting the comments for each discussion and counting them. Hiding the total while keeping the posts accessible can also break some parts of the read status and the post scrubber, which wasn't a concern of the See Past First Post extension since it also hid the posts themselves.