Scout Search

GreXXL it seems to be an issue caused by the use of a database table prefix. I'll look into this right away.

CyberGene If I search for "Chinese", I get results from other discussions and posts but not this one

Yep, that's odd. I haven't tested Algolia yet so not sure what's happening. I'll try replicating with Meilisearch this afternoon and see if it gives me any clue.

I'll need to do a proper and complete Algolia test at some point but haven't had the time yet.

CyberGene I exceeded the quota

Yep, every post (including first post of discussions), every discussion and every user will most likely count as one object. There's currently no way to exclude a specific resource but I could imagine adding an option to exclude users from Scout if this helps reduce service cost and isn't absolutely needed.

Version 0.1.2

• Fixed support for using Flarum with a database prefix
• Fixed posts not syncing to index when created through a discussion or a reply
• Fixed old content being synced when a post is edited

I recommend running the scout:import-all command to fix all existing broken records.

This update changes the way post content is indexed. Previously the markdown/bbcode version of the post from edit mode would be sent to the index. Now it's the rendered HTML version except it's stripped of any HTML. This should prevent hidden information from being leaked via search (example: old username if a mentioned user changed their username, content hidden with the hide bbcode/reply/likes extensions). But it also means image and link URLs are now no longer indexed. I will try to add an option to get those URLs indexed again in a next version.

CyberGene I was not able to replicate your issue on Meilisearch, however I found out the synchronization wasn't working correctly. So maybe your new post wasn't indexed yet.

If the problem happens again, can you try performing the search directly in Algolia's dashboard? To see if any result exists in the search index? Meilisearch has a nice built-in web interface that allows inspecting the current state of the index and I believe Algolia has something similar.

tohsakarat unfortunately it's a consequence of the current implementation, so can't be changed.

I'm not sure what the best way to calculate the sort order is. The problem is there are 2 indexes, one for discussion data (title) and one for post data (content). And I can't get the Meilisearch matching score out of Scout, only the order of the results. So merging the 2 in a way that makes sense is quite challenging. I guess it will require some specific code to go deeper level into Meilisearch API or modify the logic to use a single index.

Version 0.2.0 - TNTSearch

Following a suggestion by @010101 I have added TNTSearch support to the extension.

Existing users don't need to install this update if they are not planning to try TNTSearch.

To update:

composer require clarkwinkelmann/flarum-ext-scout

Additionally, to use TNTSearch you need to install an additional package. This package requires the sqlite PHP extension on the server!

composer require teamtnt/laravel-scout-tntsearch-driver

See the README on GitHub for more information https://github.com/clarkwinkelmann/flarum-ext-scout#tntsearch

If you switch driver, you need to rebuild the indexes with the command!

From my tests, TNTSearch seems an incredibly poor search engine. Their documentation is very lacking and the few options/syntaxes that are described don't seem to work. Maybe it can be improved slightly by playing with the settings, but I'm not sure it brings much improvement over Flarum native search.

This extension is still very experimental and designed to test alternate search engines with the goal of maybe integrating some of them natively in Flarum in the future.

fakruzaruret I'm not sure what you mean.

The 3 drivers I've tested so far are Flarum native search, Meilisearch and TNTSearch. I have still not tested Algolia myself despite the extension supporting it.

I didn't see much of a difference between Flarum native and TNTSearch. I tried TNTSearch negative keyword syntax and tried searching sentences but it doesn't seem to work. It just works with simple keywords, much like Flarum native search.

Meilisearch is still my favorite solution so far, it works great and is incredible at searching for partial words and wrong spellings. I have not had an opportunity to use it in production yet, but I'm hoping to install it on one of my forums later this year.

CyberGene where are you hosting your forum? You might be able to place Flarum and Meilisearch on the same server to save on cost.

Meilisearch seem to be working on a cloud hosting, but there's just a waitlist and no mention of the pricing. I doubt it would be affordable for this kind of use case but we never know.

It's true that if few people use the search bar, switching driver might not make a lot of sense. One place where people might not immediately realize is using the search are mentions. The auto-complete relies on the same user search endpoint. But user search is also not dramatically improved over the native search since it's just usernames that are indexed anyway.