Blomstra Search, an elasticsearch fulltext search extension

luceos

attentive being able to filter the results by discussion would be terrific too.

I think this has been discussed previously as an option for core. A more advanced search component would however be required as currently not even the gambits are visible to users (user:luceos search for instance), a component that will allow searching within discussions would therefor be needed for this change.

matteocontrini

attentive not exactly what you're asking for but this might be useful: https://discuss.flarum.org/d/32017-post-stream-search

attentive

Thank you for your responses @matteocontrini and @luceos … yeah I reckon this area will have to see attention longer term for Flarum, I don't think the core search feature is quite good enough currently. We are all loving the platform however (we're using it on our tight-knit, long-lived Arsenal forum that's been going since 2006 or so on an old PHP stack).

I am a bit tied up with my day job at the moment, and perhaps will never have time to work on this, but it seems like something that should be doable with the right combination of Elastic indexers, search criteria, etc.

llaumgui

Hi,

I using this extension for a 500k messages board. I have indexed all my content with ./flarum blomstra:search:index --recreate, since the rebuild with index creation I have no error in my logs... But I haven't any search result so. My Elastic is open on localhost:9200 and the communication between Flarum and Elastic seem OK. Is there a way to debug index / query ?

llaumgui

Hum... After running ./flarum queue:work it seem OK... I must start this like a service ?

luceos

llaumgui you should use a queue so that new posts get pushed into the elastic index. For the search to work you only need elastic to be available and have data. If you didn't see anything then perhaps you might have needed to clear the cache.

llaumgui

Other issue.

When I search and display all search results and filter on tag, response is empty. I need to refresh page to display the result. You can try this here: https://forums.fedora-fr.org/t/a-propos-de-fedora-fr?q=badge

luceos

llaumgui that might be a bug..

llaumgui

luceos llaumgui that might be a bug..

😃 Do you have a tracker ? Just for information, if I disable the extension all is OK (but slow).

luceos

llaumgui Do you have a tracker ? Just for information, if I disable the extension all is OK (but slow).

https://github.com/blomstra/flarum-ext-search I think, but it being reported here is sufficient. Just need some time to look into it.

llaumgui it depends on their implementation, but it can definitely be a search issue. We override the default endpoint so that Elastic can be fully utilized during querying.

llaumgui

Another issue, with Syndication...

Is it a Syndication issue or a Bloomstra Search issue ? Is there a way to qucikly disable Bloomstra Search for a route ?

llaumgui

luceos https://github.com/blomstra/flarum-ext-search I think, but it being reported here is sufficient. Just need some time to look into it.

Thx 😃

luceos llaumgui it depends on their implementation, but it can definitely be a search issue. We override the default endpoint so that Elastic can be fully utilized during querying.

The query builder is here: https://github.com/imorland/syndication/blob/master/src/Controller/DiscussionsActivityFeedController.php#L111 It's seem standard. Is there a way to say : "force standard search !" ?

llaumgui

llaumgui The call use the API client: https://github.com/imorland/syndication/blob/master/src/Controller/AbstractFeedController.php#L188 it's a Blomstra Search issue for my point of view.

luceos

llaumgui this extension has its own endpoint. Syndication won't be using it.. Maybe @IanM has an idea what's going on 🙏

llaumgui

luceos I look the code of Blomstra Search and I understand that your extension use specific API route:

    (new Flarum\Routes('api'))
        ->get('/blomstra/search/{type}', 'blomstra.search', Api\Controllers\SearchController::class)
        ->put('/blomstra/search/index', 'blomstra.search.index', Api\Controllers\IndexController::class)

The query used by Syndication (by @IanM) is:

$this->api->withParentRequest($request)->withQueryParams($params)->withBody($body)->withActor($actor)->get($endpoint);

I don't understand how Syndication can be use Blomstra Search... But:

if I stop my Elastic service I have a connection issue.
If I start Elastic (index is OK), I have no result.
If I disable Blomstra Search, all is OK.

llaumgui

luceos OK if I dump $last_discussions

$last_discussions = $this->getDocument($request, $actor, $params);
var_dump($last_discussions);

Result:

/var/www/forums.fedora-fr.localhost/vendor/ianm/syndication/src/Controller/DiscussionsActivityFeedController.php:116:
object(stdClass)[3080]
  public 'links' => 
    object(stdClass)[3097]
      public 'first' => string 'http://forums.fedora-fr.localhost/api/blomstra/search/discussions?sort=-lastPostedAt&filter%5Bq%5D=tag%3Aactualites&page%5Blimit%5D=10&include=lastPost%2ClastPostedUser&type=discussions' (length=185)
  public 'data' => 
    array (size=0)
      empty

api/blomstra/search seem to replace api/search/

NateS

Maybe a dumb questions, but do I need to install some elasticsearch service? The instructions made it seem like I only need to install the extension. I installed the extension, put in http://localhost:9200 for the host as described above, and get:

$ php flarum blomstra:search:index --recreate
In StaticNoPingConnectionPool.php line 64:
No alive nodes found in your cluster

EDIT: Seems a link to this would have eliminated my confusion: https://www.elastic.co/

NateS

I installed elasticsearch and it's working. I am now able to find posts that weren't showing up before, cool!

Does using this extension lose the ability to search by author? For example author:Nate returns zero results. I have more than 12,000 posts.

BTW, helpdesk.blomstra.net doesn't resolve for me.

luceos

NateS Does using this extension lose the ability to search by author? For example author:Nate returns zero results. I have more than 12,000 posts.

No that should work as well. Make sure to run the index command to seed all users to elastic.

NateS BTW, helpdesk.blomstra.net doesn't resolve for me.

Yeah, we decided it was hardly used and moved back to using just email. I will update these links in the posts once we fully changed Blomstra to Flarum BV.

NateS

luceos I ran php flarum blomstra:search:index --recreate. Before doing that the search returned nothing. Do you mean I should run something else? The same without --recreate? That pushed a bunch of stuff into the index, but author:nate still returns no results. I do have a queue:work cronjob.

I've also noticed that is:unread returns no results.

« Previous Page Next Page »