GreXXL Unfortunately I cannot provide a URL as it is not public. I am not sure what more I can provide for the setup. It is a VPS with Apache/MySQL. PHP-FPM enabled with 25 workers, OpCache enabled. Everything is the latest version. Compression is enabled.
If you request an endpoint for example here on flarum.org by going to posts/filter you can see that on flarum.org it returns in about 400-500ms, however doing the same thing on our private website takes 3-4x as long, even though the size is actually smaller (4.8 KB compared to 5.5 KB in that flarum.org example.)
As mentioned, it also does not matter if the server is experiencing high or low load.
I am really a bit stumped as to what the bottleneck could be.