Switch out backend?

BartVB

First of all; my compliments on Flarum! So far I only took a very quick look but I'm impressed with what I've seen so far! It's an open source, very good looking, performant, wel maintained, cleanly programmed and easily extendable discussion platform. Magnificent!

We are always evaluating our options when it comes to the software stack that we use and our current forum setup (phpBB 3.0) could use 'a bit' of modernisation. The new features that we are adding to our platform are built on Symfony, and the longer you look at Symfony the uglier viewtopic.php gets ?

If we decide to go with Flarum we could of course convert everything to Flarum and celebrate. The idea of doing that really gives me the creeps though... Our long time users are extremely used to the current layout/categories/forums/features/paging. We also have quite a few admin tools which are based on phpBB and we know that phpBB can handle the current load. As far as I can tell Flarum hasn't been load tested or even optimised yet. It would be really sweet if we could slowly transition the site from phpBB to Flarum.

For us the biggest appeal of Flarum is it's frontend, which made me wonder if it would be a viable option to create a different API backend for Flarum which wouldn't talk to Doctrine but to the phpBB database.

As I said, I only briefly browsed the code, so I have no idea how intertwined the frontend and backend are. Would this be a viable option? And, if yes, what would be needed to get that show on the road?

Davis

It's posible. Although, in my opinion most of the bugs we have are for the front end, so you won't be avoiding everything. It defendantly would be some work too. About load testing: it runs on servers wonderfully not using many resources. I've seen it load at 300ms with caching. I wouldn't say flarum is going to have any performance issues (that's the last thing I personally would worry about). Just look at our forum here. It's been working pretty flawlessly. Couple bugs here and there and a general lack of features, but it's feasable. The most important thing here would be constant database backups. Although, flarum comes with no warranty so use at your own risk and running it on a production environment is not recomended (officially).

BartVB

If there is a usable and complete API it should be possible to use our current system as the backend and slowly migrate features that we are missing. That would benefit both Flarum and us. Going the API route allows us to keep the old interface up and running for our old time users that tend to be very resistant to change. This would also leave us with a 'fallback' option if Flarum doesn't fare well with our users or with the complexity of our site. And we know what our current backend can handle load-wise.

I know the software has been working very well on this site, but we need it to scale to 300k users, 2M topics and 100M posts. It should be possible to do that with every reasonably designed forum system do I'm confident that Flarums backend will be able to handle this. But for now there is too big a gap between our current setup and Flarum when it comes to performance and features.

I'll see if I can block some time next week to assess what would be needed to make this happen.

Davis

You're confusing me. First you want to use the phpbb backend, now you want to keep the old look for users that are resiliant from change. What exactly is it you want here? And I'm not sure developing for only one half of the flaurm setup would really benifit anyone but yourself. I believe @webeindustry has done some performance testing on flarum and can tell you about it in specifics.

BartVB but for now there is too big a gap between our current setup and Flarum when it comes to performance and features

What? Flarum is very lightweight. I don't understand people's automatic assumption that because we're in beta it is slow and has performance issues. Granted some bugs pop up but the initial payload is very small, the API fetches only the data it needs after that.

BartVB

Davis Sorry, I wasn't very clear in my post. I didn't really specify when I was talking about the Flarum frontend and backend.

The idea that I'm currently entertaining is keeping the phpBB database as it is and writing an API bridge for the Flarum frontend. Depending on how Flarum is organised I probably won't be using much of the Flarum backend at this point.

The Flarum frontend would be the main interface of our forum, this would talk to the API bridge which updates the phpBB database. Because we won't change the database the old interface will remain functional, including all the extra moderation tools.

When all that is up and running we can extend the Flarum frontend with features that we port from our old forum. As you said; a big part of Flarum is in the frontend, so if we port something to the frontend it will be fairly easy to make that feature available in a 100% Flarum setup by writing a fairly small piece of Flarum backend. That would be a win-win. Especially if we would sponsor you to write the frontend (and maybe also the backend) part(s).

Regarding performance; I'm certainly not assuming that Flarum is slow and has performance issues because it's in beta. I haven't done any real testing yet, but according to the issuetracker and roadmap the database doesn't have any indices yet except for the primary keys. It's close to impossible to get decent performance out of that on the backend side of things. That's probably reasonably easy to solve, but it's doesn't seem to be tested/implemented yet. The frontend is rather responsive indeed. I would like < 100ms for every PHP call, but the current state already seems to be very acceptable, also because the interface is decoupled from the backend. And the frontend certainly seems to be snappy. Haven't tested older mobile devices yet, but it seems to be faster than most 'webapps' that I've seen. Kudos on that!

isti37 How would mysql-live-select fit into an API bridge? Live select seems to be focussed on sending streaming data like statistics or newsfeeds. Flarum would need random access to the whole database.
But very curious as to your plans, you seem to have experience with larger phpBB installs?

webeindustry

@BartVB I have done some load-testing, and can say how many requests per second a server of specific specs can be handled on Flarum as-is. On a DO droplet we're looking at 2-3 requests per second to the home page of discuss. With our 6 core beta server we're hitting 40 requests per second with errors starting to pop up at 160 concurrent user requests. Our production gear has dual socket 8 core xeons that handle 90-100 requests per second and about 360 concurrent requests. We plan on greatly increasing this capacity when Flarum is properly adapted to handle Nginx caching.

I don't think Flarum can keep up with older forum systems, but it can take on quite a bit more than Discourse given the same hardware in place.

isti37

We intend to do the same thing on a forum with 100k members.
My current idea is to use mysql-live-select https://github.com/numtel/reactive-mysql-example and websockets with node or akka.io backend. However mysql-live-select is based on the binary logs of mysql. There is no way someone could use mysql-live-select and sockets without a private server.

FlarumBB is made so it can run on cpanel. I think that anything more advanced is out of the scope of flarum. Indeed we could copy the interface and use websockets with reactive mysql with the tables of phpbb/vbulletin/ipb and even flarum for a reactive frontend, but that is totally different from a simple forum software.

If there is interest I might pursue developing a reactive frontend for phpbb/ipb/vbulletin.

jordanjay29

BartVB I don't think what you're asking is really possible. You're basically thinking about ripping out the guts of Flarum, putting in a phpBB database, and expecting to match up the front end features to that. Considering Flarum has several workflows that are different from phpBB's, the end result for you isn't likely to be pretty. You're basically trying to surgically attach a monkey's head to a pig's body, and expecting it to live.

I understand what you're trying to do. You want a failsafe, I get it. You want the same features that you and your users are used to, I get it. My best advice to you is this:
1. Help us perfect the phpBB->Flarum conversion process. Either by trialing the phpBB importer in Resources with a copy of your board's database or diving into the code and hacking away at it yourself.
2. Help design or create extensions that will close the feature gap between phpBB's moderation functions and Flarum's.
3. Have patience. Flarum is in beta, and there are still features yet to come that will likely bridge what gaps remain, and other extension authors who will step up to create those that aren't included by default.
4. When the time comes, convert completely. Make your users aware of what features they may keep and gain (there will likely be some they like better than phpBB, and some people who will always prefer phpBB's way). Then keep your phpBB backup, but don't look back unless it's seriously not working for you. I promise you, users will get used to the new way of doing things, and if you need a feature you used to have there's always extensions.

BartVB

The API seems to be extremely compatible with phpBB and similar forum systems:
http://flarum.org/docs/api/

There are topics, posts, users, groups, permissions. Notifications and tags would need some extra work, but that seems to be very doable. But when it comes to implementing this the question is if the backend and frontend are cleanly separated by the API?

I get that from your perspective migrating to Flarum frontend + backend would be the better course of action, but we have an enormous investment in phpBB which makes a full migration hard and extremely risky for us. We like baby-steps when it comes to big changes like these ?

For now we are only evaluating our options. This will not be a short term project for us. But if this would work it could be an interesting 'best of both worlds' solution.

Davis

BartVB but we have an enormous investment in phpBB which makes a full migration hard and extremely risky for us.

This doesn't make any sense. Just back up the database and experiment with it elsewhere. If you can migrate successfully, copy the files over to production. If there's any missing features you're best developing them for flarum instead of this proprietary system you want to create.

Franz

BartVB Yes, the backend and frontend are pretty much completely separated, coupled only by the API and the (fairly) light-weight layer that generates the HTML for the single-page app.

However, you are probably going to hit some problems with extensions, because many of the features that you can see in the interface of discuss.flarum.org are provided by extensions - not sure how you would handle the API parts of those vs. the core...

luceos

I agree with Franz on this one. Another issue I can immediately point you at is the following;

it will take time to create a fully working intermediary layer between Flarum frontend and phpBB
during beta, Flarum will still experience drastic changes. Speaking from personal experience, some time ago how Extensions were handled was completely rewritten and it's still continuously evaluated.
it will take a lot of effort to make the phpBB features/extensions do anything in the frontend.

My advice, start working on extensions for Flarum if you find the project interesting. Otherwise stick to phpBB.