IanM This is very close to something I've already built and have been running on my forum for a couple of weeks now. So let me share some practical experience, hopefully it's useful for the guardrails part.
Mine is a mention-based bot: a member mentions a designated bot account in a discussion and Claude replies in the thread as that account. It's queue-backed and it builds a token-budgeted context out of the thread. It also has allow/deny lists for users, groups and tags, a per-user and a forum-wide daily cap, and an audit table for the spend.
So it's basically the "reply/draft assist" row in your table, which you put in the ecosystem, and I fully agree that's where it belongs. However, roughly a third of the code I ended up writing is exactly the plumbing you're proposing for flarum/ai. I'd be very happy to throw that third away.
A few things I learned the hard way, in no particular order.
The settings table is not a safe place for an API key, and it's not obvious why. Flarum's AdminPayload serializes the whole settings table into the admin page HTML. So a key stored there is sitting in the DOM, readable by every admin, but also by every other extension's admin JavaScript. I had to write a specific listener that masks the value before it's serialized.
Encrypting at rest doesn't help by itself here, because the payload gets the decrypted value anyway. And flarum/ai certainly should bridge keys into the admin UI, since not everyone installing from Packagist has shell access to edit config.php. But then this has to be solved once in the foundation, otherwise every feature extension inherits the same hole.
Quotas and cost ceilings have to be counted from durable rows, not from a counter. My first attempt was a cache counter and it's simply wrong, because a worker restart, a deploy or a queue drain hands everyone a fresh allowance. I count rows in the audit table over a rolling 24h window instead, which is one indexed COUNT.
That has a nice side effect too. The quota becomes queryable and not only enforceable, so I can warn the member in the composer, before they submit, that their post won't get an answer. IMO that "tell the user before they spend" surface belongs in the foundation as well, because every feature will need it.
Queue-by-default needs to be an explicit refusal on the sync driver, not just a dispatch. Dispatching to a queue that is on the sync driver does nothing. The API call then blocks the web request and the user is staring at a spinner for half a minute. My extension simply refuses to run and says so in the admin. A lot of small self-hosted forums are on the sync driver without knowing it.
One more thing, which is more of a design opinion. My gate is per-tag and not only per-user/group, and that turned out to be more important than I expected. It's how you keep the bot out of your Politics tag, or off private discussions. A single user.use-ai permission can't express that. So I'd suggest the foundation's gate takes the discussion and its tags into account from day one, otherwise every feature will reinvent it slightly differently, and inconsistently.
And a small caveat about the provider-agnostic layer. I use a couple of Anthropic-specific things that I doubt would survive an abstraction: a pre-flight token count, so I can refuse a request before it's billed, and their server-side web search tool. Some escape hatch down to the underlying provider client would be very welcome. Otherwise extensions that need such things will just keep talking to the vendor SDK directly and will lose all the guardrails, which kind of defeats the purpose 🙂
Anyway, +1 from me on the general shape, and especially on Ollama being first class. That's the thing that makes AI features acceptable for a lot of self-hosted forums. I might be missing something of course, and my extension is Claude-built as usual, so take it for what it is. Happy to share code or more details on any of the above if it's useful.
(The above is something I wrote mixed with Claude's details on the extension and I asked it to not change my tone/language, I'm not a native language speaker, but the grammar is fixed, so a bit too perfect than usual for me 🤣)
