This report is prepared by Claude, ive been running claude on my vps it has used almost 20m token to build my forum and is very good at catching my custom extension's errors and logs
so i thought maybe you guys can help me with this problem in a way proffesional way. currently i dont want to upgrade to a 4vcpu vps because i believe i dont need it.
whats your thoughts.
flarum/pusher: every viewer refetches the whole discussion on every new post — O(viewers × posts) during a burst
Setup
- Flarum 2.x, 8,700 registered users
flarum/pusher v2.0.0-rc.4, pointed at a self-hosted Soketi instance (Pusher wire-compatible)- 2 vCPU / 4 GB VPS, Caddy + PHP-FPM 8.3 + MySQL + Redis
We run a scheduled community event (a "boss fight") where 50 members are on
the site at the same time. When it ends, a bot opens a fresh results topic
and everyone opens it within about a minute. That minute is where the server
falls over.
Symptom
During the post-event rush, response times across the whole site degrade —
not just the busy topic. Unrelated pages (the member map, other endpoints)
slow down too, because PHP-FPM workers are saturated.
Notably, MySQL is idle throughout: Threads_running 2, Slow_queries 0.
The bottleneck is PHP request volume, not query cost.
Measurement
Aggregated from Caddy access logs over the event window:
| Endpoint | Requests | Avg | p95 | Total time | Avg size |
|---|
/api/discussions/{id} | 2,432 | 4,938 ms | 12,630 ms | 12,009 s | 2.4 KB |
/api/posts | 930 | 4,510 ms | 12,869 ms | 4,194 s | 3.9 KB |
/api/discussions | 1,212 | 2,321 ms | 11,105 ms | 2,813 s | 6.5 KB |
| (our own extension's endpoint) | 4,751 | 440 ms | 1,765 ms | 2,093 s | 0.5 KB |
/api/discussions/{id} alone accounts for more server time than everything
else combined. Note the shape: our own endpoint received twice the
requests but consumed a sixth of the time.
The topic itself was small — 14 posts, created one minute earlier — and
the response is only 2.4 KB. So this isn't a "huge thread" problem.
I should be upfront that the 4.9 s average is partly self-inflicted: once
FPM saturates, everything queues, which inflates the numbers further. Under
no load the same call is roughly 150 ms. It's a feedback loop, not a single
slow query.
Mechanism
From flarum/pusher/js/src/forum/index.tsx:
extend(DiscussionPage.prototype, 'oncreate', function () {
app.pusher.then((binding) => {
binding.pusher.bind('newPost', (data) => {
const id = String(data.discussionId);
const discussionId = this.discussion?.id();
if (this.discussion && discussionId === id && this.stream) {
const oldCount = this.discussion.commentCount() ?? 0;
app.store.find('discussions', discussionId).then(() => {
this.stream?.update().then(m.redraw);
// ... unread title count
});
}
});
});
});
For every new post, every client with that discussion open performs:
app.store.find('discussions', id) — a full discussion refetchstream.update() — which then fetches posts
With V viewers and M posts arriving, that's V × M discussion fetches
plus V × M stream updates. In our case 20 viewers × 14 posts ≈ 280 + 280,
which matches the measured 2,432 across the wider window.
Because the burst is caused by the same people who are watching it, the
traffic scales with the square of engagement — the more popular the moment,
the worse it gets.
Why step 1 looks redundant
stream.update() already fetches the new posts. The discussion refetch
appears to exist mainly to refresh commentCount() for the unread
title-count and the scrubber total.
If that's correct, the expensive half of the work is being done for a counter.
What we've considered, and why each is unsatisfying
Debounce the refetch from our own extension. The handler is an anonymous
function, so it can't be removed selectively — we'd have to
pusher.unbind('newPost'), which removes every extension's listener, then
rebind ours plus a throttled reimplementation of core's. That means our
extension silently owns a piece of core realtime behaviour and breaks on any
upstream change.
Throttle app.store.find for discussions. We counted 14 call sites using
that signature across core and extensions; throttling would affect legitimate
callers that need fresh data.
Patch the extension directly. Works, but reverts silently on
composer update — and a performance regression that reappears months later
is very hard to attribute.
More CPU. Helps, but the request pattern would still be wasteful.
Questions
- Is the discussion refetch necessary, or could
commentCount be derived
from the stream update / carried in the newPost payload (which already
includes postId, discussionId, tagIds)?
- Would a short debounce (collapsing bursts into one refresh every few
seconds) be acceptable upstream? Replies would still appear promptly —
arguably faster than today, since the current behaviour is what causes
the queueing.
- Is there a supported extension point to modify this behaviour without
unbinding every other newPost listener?
Happy to test a patch against a real event — we run these regularly and can
reproduce the load reliably, with before/after log analysis.
