Huge performance problem - Refreshing takes 20sec or longer
You can change the settings
table key extensions_enabled
, as long as it's valid json
array it should be safe to do.
Okay I found the problem.
"Likes" Extension is causing the massive latency on refresh.
We have around 440 members and around 430k likes.
I had a similiar issue once with the Like Counter extension beta: https://discuss.flarum.org/d/23878-like-counter-extension-beta/36 from @johnlewissims.
@askvortsov identified it and I had to switch that extension off.
@clarkwinkelmann's extension Likes Received is what I am using now which raised some mariaDB update errors on my server: https://discuss.flarum.org/d/24489-likes-received/31 But I am fine staying at the current mariaDB version.
Back to the Likes-Extension: what can I do? If I switch it off and refresh a discussion with more than 10k posts it is getting refreshed immediately. Community really loves it so switching it off is no option I guess.
- Edited
m4v3rick "Likes" Extension is causing the massive latency on refresh.
This is the flarum/likes extension? If so we need to investigate this as soon as possible and patch it.
Never mind, the issue seems obvious. For a discussion every like is always loaded, with a very vast amount of likes this indeed won't perform well. I'll push this to the top of our todo.
Guess that's a no: https://discuss.flarum.org/d/27852-flarum-v1-release-announcements/17
Any news to this issue?
My apologies for butting in a conversation I know nothing about but this line, is it run whenever a post is saved or when its data is sent? If the extension runs a grouping query per post then you could easily run into catastrophic performance issues once the number of rows exceed the amount of free RAM. That's not something you'd notice during development, unless your test board is on a small VPS of sorts.
JoshyPHP the code you linked is not compatible with Flarum 1.x, but yes it would have run one time for each post returned by the REST API or part of the preloaded payload.
Oh right, I was reading from the wrong extension's repository.
Your extension stores the number of likes in the users
table, so of course it shouldn't have any measurable impact on SQL performance.
What happens if you only disable these two extensions:
nearata-no-self-likes
clarkwinkelmann-likes-received
It's a known issue that the flarum/likes
extension doesn't scale well, because it loads the list of all likers for each post.
However we have yet to find out exactly how bad it is, and what the best solution is. I have personally not troubleshooted this and based on the earlier replies above, it seems like those who tried replicating didn't have a good enough data set.
The easiest solution would be to only compute the total amount of likes, save it in the database, and then show that number on the post instead of the list of people who liked. Currently the database structure and code for this doesn't exist. Then the existing relationship for likers would have to be modified to only return a subset of the likers if we want to preserve the same UI as currently. And finally a new API endpoint would have to be added to load the list of likers on demand if we want to preserve that feature when clicking "More".
The easiest short-term solution would be to create a fork of the likes extension that drops the likers and only shows the total. It would be the easiest implementation. I don't think we should drop the list of likers from the core extension, it's a nice UX of the current extension. But this could then become an option in the Likes extension so that high activity forums can hide the list of likers entirely to improve performance further.
My "likes received" extension already does the pre-calulation of like count, but only for user profiles, it doesn't change how likes are counted and returned for individual posts.
I have actually been working on a tentative attempt to fix it (https://github.com/flarum/framework/compare/dk/likes/load-only-necessary-likes?expand=1), however I will need to test this against a dataset that's large enough to notice the changes in performance before this can be reviewed. As such this won't be in the upcoming release yet and I'm careful stating it will be in the one thereafter as it might contain backward breaking changes..
- Edited
Wellwisher clarkwinkelmann-likes-received
That one I already disabled because the maria DB update broke the refreshing command. Dunno why but I already discussed it in another thread.
Haven't tried no-self-likes and will give it a shot.
Edit: Just tried it, but didn't help.
m4v3rick I'm not saying it isn't the likes extension btw. Just that based on the discussions linked it shouldn't be a problem with loading all likes of a post, but probably something else.
m4v3rick Well the first link i posted has 5k posts and almost 12k likes. Is that little to no likes?
I'm guessing 12k is across all posts of the discussion? you have to look at the posts visible when the discussion is loaded. which is about 20 posts, and across all first loaded 20 posts, there seem to be less than 20 likes.
To better diagnose the issue it would be best to install fof/clockwork
and take a look at what's happening. Hit meup on discord if you need help on how to share the right clockwork produced information (should make a tutorial or smth cause it's not straightforward to use)
Installed it and added the line in the .nginx.conf.
When I open the discussion now from Discussion site it works. If I use the link directly I get an error 500.
If I switch the extension off and comment the .nginx.conf lines out the discussion loads again with the direct link but takes again ages. Switching like extension off - everything runs ultra smooth.
But if it's possible we could check it via discord. I contacted you.
I updated to the newest flarum version which worked great and also OS updates are done.
- Edited
m4v3rick If I use the link directly I get an error 500.
If I switch the extension off and comment the .nginx.conf lines out the discussion loads again with the direct link but takes again ages. Switching like extension off - everything runs ultra smooth.
If others have the same issue. The 500 error was because of the massive load. Setting php.ini memory_size up to 2G helped and clockwork works, so @SychO could analyze the issue. Thanks for your help again.
Seems like the SEO extension triggers the likes extension. So the like extension isn't the problem source .