Blomstra Search, an elasticsearch fulltext search extension

luceos

Teddan yeah absolutely. I will need to find a solution to that, but I think it's similar to the fix of core.

Teddan

luceos Thank you!! Looking forward to that !

Hari

where can I test this extension without installing it, Is there any website already using it?

i have checked previous posts and went to https://sorhadi.net/ to check
i picked a line from this discussion
Yediğimizi mide hazmederken karın ağrısı when i search using this search not showing any result.

Does flarum or blomstra/search supports full-sentence search?

https://imgur.io/a/LwziYgK

luceos

Hari try this site; forum.mutluanneleriz.com please don't link, https://blomstra.community also has it running but that dataset is much less. Mutluanneleriz contains gigabytes on data in elastic.

Hari

matteocontrini i have almost 30k discussions and it will grow to 70k discussions in the next 10 months. we are using our flarum as a backend application to address user needs, most users will not come to my flarum and perform actions only the admins or mods use it to find similar threads and try to post existing answers so i do not think there will be much user search queries but i am sure this will change after one year. (i will give an interview on this to gXXL after few months, it is a unique case study how we are using flarum)

like mentioned here Hari i am looking for full-sentence matches in post content search

i have 4GB RAM with 2vCPU Plesk setup running for only flarum and 2-3 WordPress subdomains.

do you recommend meilisearch for my need? since you have explored all the possible ways you are the right person to get suggestions from. 🙂

Hari

luceos i have taken this line kaydeden koleksiyonun adını yazsın kaydedeyim and searched, the search is not showing the below mentioned disucssion

forum.mutluanneleriz.com/?q=kaydeden%20koleksiyonun%20adını%20yazsın%20kaydedeyim
forum.mutluanneleriz.com/d/798517-trendyol-koleksiyon-sadece-burada-paylasabilirsiniz/11

luceos

Hari looks like it.

There's definitely room for more tuning and matching mechanisms. I also think it could do with some unit tests. Previous releases of this extension have been mostly about getting data into elastic, improving performance for seeding massive datasets and showing some sensible results.

avandorp

Nice plugin. Is it possible to index uploaded PDF files (e.g. uploaded via FoF/Upload extension) this way? Or is there another way to do that?

luceos

avandorp it is possible, but getting there is not done with the flick of a hand. Flarum itself has no mechanism yet to search files, so that would be step 1. Then one would need to sync uploads (or rather their information) to elastic.

avandorp

Thanks for the fast answer. Naively I imagined that in uploader I would hook up to the File/WasSaved event and via an ElasticSearch ingester I would put the pdf content into the search index. But being new to Flarum I don't see where I would get the info of where the search would have to point to. Where in Flarum would be the right place to do so?

luceos

avandorp Flarum has Search Sources in js. This source would then be added to search, in addition this search source then would need an endpoint that queries elastic for related resources.

Ref:

DiscussionSearchSource: https://github.com/flarum/framework/blob/main/framework/core/js/src/forum/components/DiscussionsSearchSource.tsx
DiscussionSearch endpoint: https://github.com/flarum/framework/blob/main/framework/core/src/Api/Controller/ListDiscussionsController.php

Also look at how I extended DiscussionSearch endpoint: https://github.com/blomstra/flarum-ext-search/blob/main/src/Api/Controllers/SearchController.php
Here we are replacing the old search source, you can mimic this by just adding your own files source: https://github.com/blomstra/flarum-ext-search/blob/aabba9c2d0ff90a15be9ddcd1cc257a497d45607/js/src/forum/index.ts#L16

meihuak

does it work in Chinese forum?

luceos

meihuak it should actually work just fine.

meihuak

luceos

You can try to search for Chinese words, visit this link, copy the "测试中文搜索功能" to your forum, and see if you can only search for a single Chinese word, for example, "孔子" only searches for "孔", "曹操" searches for "" 曹" or "操", "blomstra/search" used to be able to search for phrases, but after the latest update, only a single word can be searched.

luceos

meihuak exactly. So.. have you tried it?

attentive

Hi there @luceos this looks like a really useful extension and I am somewhat familiar with setting up Elasticsearch so that's maybe a positive too. Thank you 👏

We have a new Flarum setup we're really happy with, but we'd like to get search to return posts as results, rather than discussions—if it's just discussions, then it's hard to find the right post. Does Flarum built in search or does your Blomstra Search extension support listing search results as posts?

luceos

attentive it's not possible out of the box, but it could most certainly be added to either. Let me know if you need further help.

matteocontrini

attentive not exactly what you're asking for but this might be useful: https://discuss.flarum.org/d/32017-post-stream-search

attentive

Do you mean "added" as in by forking and altering the code? I could maybe try to do that, I'm a software contractor although PHP is not my thing.

The reason I ask is that the leading complaint on my newly migrated bulletin board (used to be MyBB) is that when you search for a keyword with the out of the box feature, you can't get a list of posts as the results.

Since we have a heap of 1,000+ post discussions with repeated keywords, this is a bit of a pain … being able to filter the results by discussion would be terrific too.

luceos

attentive being able to filter the results by discussion would be terrific too.

I think this has been discussed previously as an option for core. A more advanced search component would however be required as currently not even the gambits are visible to users (user:luceos search for instance), a component that will allow searching within discussions would therefor be needed for this change.

attentive

Thank you for your responses @matteocontrini and @luceos … yeah I reckon this area will have to see attention longer term for Flarum, I don't think the core search feature is quite good enough currently. We are all loving the platform however (we're using it on our tight-knit, long-lived Arsenal forum that's been going since 2006 or so on an old PHP stack).

I am a bit tied up with my day job at the moment, and perhaps will never have time to work on this, but it seems like something that should be doable with the right combination of Elastic indexers, search criteria, etc.

« Previous Page Next Page »