Scout Search

clarkwinkelmann

Hari I'm not super familiar with Algolia so not really sure about the best approach here.

I don't think you can use a single unified index for Flarum stuff and another app, as scout requires its own dedicated index for each resource. But maybe Algolia has ways to merge data from multiple indexes when performing a search via the API, not sure.

GreXXL

clarkwinkelmann tried on flarumde for fun with Algolia. Importing the data to Algolia worked like a charm But I am receiving 500 errors when trying to search.

request.js:170 GET https://www.flarumde.com/api/discussions?filter%5Bq%5D=ank&page%5Blimit%5D=3&include=mostRelevantPost 500 (Internal Server Error)

Next Illuminate\Database\QueryException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'matching_posts.priority' in 'field list' (SQL: select `flarumde_discussions`.*, COALESCE(posts_ft.most_relevant_post_id, discussions.first_post_id) as most_relevant_post_id from `flarumde_discussions` left join (select `flarumde_posts`.`discussion_id`, id as most_relevant_post_id from `flarumde_posts` inner join (select `flarumde_posts`.`discussion_id`, min(matching_posts.priority) as min_priority from `flarumde_posts` inner join (select `flarumde_posts`.`discussion_id`, FIELD(id, 67, 179, 209, 161, 7) as priority from `flarumde_posts` where exists (select 1 from `flarumde_discussions` where `flarumde_discussions`.`id` = `flarumde_posts`.`discussion_id` and (`flarumde_discussions`.`id` not in (select `discussion_id` from `flarumde_discussion_tag` where `tag_id` not in (select `flarumde_tags`.`id` from `flarumde_tags` where (`flarumde_tags`.`id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) and (`flarumde_tags`.`parent_id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) or `flarumde_tags`.`parent_id` is null))))) and (`flarumde_discussions`.`is_private` = 0 or (((`flarumde_discussions`.`is_approved` = 0) or (`flarumde_discussions`.`id` in (select distinct `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null and (`flarumde_recipients`.`user_id` = 1 or `flarumde_recipients`.`group_id` in (1, 5))) or (`flarumde_discussions`.`id` in (select `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null) and `flarumde_discussions`.`id` in (select distinct `flarumde_posts`.`discussion_id` from `flarumde_posts` where `flarumde_posts`.`id` in (select distinct `flarumde_flags`.`post_id` from `flarumde_flags`)))))))) and (`flarumde_posts`.`is_private` = 0 or (((`flarumde_posts`.`is_approved` = 0)))) and `flarumde_posts`.`type` = comment and `id` in (67, 179, 209, 161, 7) and `type` in (discussionTagged, discussionSplit, discussionStickied, discussionLocked, recipientLeft, recipientsModified, madePublic, discussionMerged, comment, discussionRenamed)) `flarumde_matching_posts` on `flarumde_matching_posts`.`discussion_id` = `flarumde_posts`.`discussion_id` where `type` in (discussionTagged, discussionSplit, discussionStickied, discussionLocked, recipientLeft, recipientsModified, madePublic, discussionMerged, comment, discussionRenamed) group by `flarumde_posts`.`discussion_id`) `flarumde_best_matching_posts` on `flarumde_best_matching_posts`.`discussion_id` = `flarumde_posts`.`discussion_id` where exists (select 1 from `flarumde_discussions` where `flarumde_discussions`.`id` = `flarumde_posts`.`discussion_id` and (`flarumde_discussions`.`id` not in (select `discussion_id` from `flarumde_discussion_tag` where `tag_id` not in (select `flarumde_tags`.`id` from `flarumde_tags` where (`flarumde_tags`.`id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) and (`flarumde_tags`.`parent_id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) or `flarumde_tags`.`parent_id` is null))))) and (`flarumde_discussions`.`is_private` = 0 or (((`flarumde_discussions`.`is_approved` = 0) or (`flarumde_discussions`.`id` in (select distinct `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null and (`flarumde_recipients`.`user_id` = 1 or `flarumde_recipients`.`group_id` in (1, 5))) or (`flarumde_discussions`.`id` in (select `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null) and `flarumde_discussions`.`id` in (select distinct `flarumde_posts`.`discussion_id` from `flarumde_posts` where `flarumde_posts`.`id` in (select distinct `flarumde_flags`.`post_id` from `flarumde_flags`)))))))) and (`flarumde_posts`.`is_private` = 0 or (((`flarumde_posts`.`is_approved` = 0)))) and `id` in (67, 179, 209, 161, 7) and FIELD(id, 67, 179, 209, 161, 7) = best_matching_posts.min_priority and `type` in (discussionTagged, discussionSplit, discussionStickied, discussionLocked, recipientLeft, recipientsModified, madePublic, discussionMerged, comment, discussionRenamed)) `flarumde_posts_ft` on `flarumde_posts_ft`.`discussion_id` = `flarumde_discussions`.`id` where (`flarumde_discussions`.`id` not in (select `discussion_id` from `flarumde_discussion_tag` where `tag_id` not in (select `flarumde_tags`.`id` from `flarumde_tags` where (`flarumde_tags`.`id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) and (`flarumde_tags`.`parent_id` in (select `flarumde_perm_tags`.`id` from `flarumde_tags` as `flarumde_perm_tags`) or `flarumde_tags`.`parent_id` is null))))) and (`flarumde_discussions`.`is_private` = 0 or (((`flarumde_discussions`.`is_approved` = 0) or (`flarumde_discussions`.`id` in (select distinct `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null and (`flarumde_recipients`.`user_id` = 1 or `flarumde_recipients`.`group_id` in (1, 5))) or (`flarumde_discussions`.`id` in (select `flarumde_recipients`.`discussion_id` from `flarumde_recipients` where `flarumde_recipients`.`removed_at` is null) and `flarumde_discussions`.`id` in (select distinct `flarumde_posts`.`discussion_id` from `flarumde_posts` where `flarumde_posts`.`id` in (select distinct `flarumde_flags`.`post_id` from `flarumde_flags`))))))) and (`most_relevant_post_id` is not null or 0 = 1) group by `flarumde_discussions`.`id` order by FIELD(id, 67, 179, 209, 161, 7) limit 4) in /path/httpdocs/vendor/illuminate/database/Connection.php:712
Stack trace:
#0 /path/httpdocs/vendor/illuminate/database/Connection.php(672): Illuminate\Database\Connection->runQueryCallback()
#1 /path/httpdocs/vendor/illuminate/database/Connection.php(376): Illuminate\Database\Connection->run()
#2 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2414): Illuminate\Database\Connection->select()
#3 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2402): Illuminate\Database\Query\Builder->runSelect()
#4 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2936): Illuminate\Database\Query\Builder->Illuminate\Database\Query\{closure}()
#5 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2403): Illuminate\Database\Query\Builder->onceWithColumns()
#6 /path/httpdocs/vendor/illuminate/database/Eloquent/Builder.php(625): Illuminate\Database\Query\Builder->get()
#7 /path/httpdocs/vendor/illuminate/database/Eloquent/Builder.php(609): Illuminate\Database\Eloquent\Builder->getModels()
#8 /path/httpdocs/vendor/flarum/core/src/Search/AbstractSearcher.php(68): Illuminate\Database\Eloquent\Builder->get()
#9 /path/httpdocs/vendor/flarum/core/src/Api/Controller/ListDiscussionsController.php(100): Flarum\Search\AbstractSearcher->search()
#10 /path/httpdocs/vendor/flarum/core/src/Api/Controller/AbstractSerializeController.php(115): Flarum\Api\Controller\ListDiscussionsController->data()
#11 /path/httpdocs/vendor/flarum/core/src/Http/RouteHandlerFactory.php(41): Flarum\Api\Controller\AbstractSerializeController->handle()
#12 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ExecuteRoute.php(27): Flarum\Http\RouteHandlerFactory->Flarum\Http\{closure}()
#13 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ExecuteRoute->process()
#14 /path/httpdocs/vendor/kilowhat/flarum-ext-audit-pro/src/Middlewares/ExtendSetPermissionController.php(19): Laminas\Stratigility\Next->handle()
#15 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Kilowhat\Audit\Middlewares\ExtendSetPermissionController->process()
#16 /path/httpdocs/vendor/kilowhat/flarum-ext-audit-pro/src/Middlewares/SetLoggerActor.php(28): Laminas\Stratigility\Next->handle()
#17 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Kilowhat\Audit\Middlewares\SetLoggerActor->process()
#18 /path/httpdocs/vendor/flarum/core/src/Api/Middleware/ThrottleApi.php(33): Laminas\Stratigility\Next->handle()
#19 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Api\Middleware\ThrottleApi->process()
#20 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/CheckCsrfToken.php(36): Laminas\Stratigility\Next->handle()
#21 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\CheckCsrfToken->process()
#22 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ResolveRoute.php(67): Laminas\Stratigility\Next->handle()
#23 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ResolveRoute->process()
#24 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/SetLocale.php(51): Laminas\Stratigility\Next->handle()
#25 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\SetLocale->process()
#26 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/AuthenticateWithHeader.php(58): Laminas\Stratigility\Next->handle()
#27 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\AuthenticateWithHeader->process()
#28 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/AuthenticateWithSession.php(31): Laminas\Stratigility\Next->handle()
#29 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\AuthenticateWithSession->process()
#30 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/RememberFromCookie.php(52): Laminas\Stratigility\Next->handle()
#31 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\RememberFromCookie->process()
#32 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/StartSession.php(61): Laminas\Stratigility\Next->handle()
#33 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\StartSession->process()
#34 /path/httpdocs/vendor/flarum/core/src/Api/Middleware/FakeHttpMethods.php(29): Laminas\Stratigility\Next->handle()
#35 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Api\Middleware\FakeHttpMethods->process()
#36 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ParseJsonBody.php(28): Laminas\Stratigility\Next->handle()
#37 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ParseJsonBody->process()
#38 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/HandleErrors.php(57): Laminas\Stratigility\Next->handle()
#39 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\HandleErrors->process()
#40 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/InjectActorReference.php(25): Laminas\Stratigility\Next->handle()
#41 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\InjectActorReference->process()
#42 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(76): Laminas\Stratigility\Next->handle()
#43 /path/httpdocs/vendor/middlewares/request-handler/src/RequestHandler.php(84): Laminas\Stratigility\MiddlewarePipe->process()
#44 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\RequestHandler->process()
#45 /path/httpdocs/vendor/middlewares/base-path-router/src/BasePathRouter.php(101): Laminas\Stratigility\Next->handle()
#46 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\BasePathRouter->process()
#47 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Middleware/OriginalMessages.php(36): Laminas\Stratigility\Next->handle()
#48 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Laminas\Stratigility\Middleware\OriginalMessages->process()
#49 /path/httpdocs/vendor/middlewares/base-path/src/BasePath.php(73): Laminas\Stratigility\Next->handle()
#50 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\BasePath->process()
#51 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ProcessIp.php(24): Laminas\Stratigility\Next->handle()
#52 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ProcessIp->process()
#53 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(76): Laminas\Stratigility\Next->handle()
#54 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(65): Laminas\Stratigility\MiddlewarePipe->process()
#55 /path/httpdocs/vendor/laminas/laminas-httphandlerrunner/src/RequestHandlerRunner.php(96): Laminas\Stratigility\MiddlewarePipe->handle()
#56 /path/httpdocs/vendor/flarum/core/src/Http/Server.php(44): Laminas\HttpHandlerRunner\RequestHandlerRunner->run()
#57 /path/httpdocs/public/index.php(26): Flarum\Http\Server->listen()
#58 {main}  
[2022-06-27 11:57:31] flarum.ERROR: PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'matching_posts.priority' in 'field list' in /path/httpdocs/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php:82
Stack trace:
#0 /path/httpdocs/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOConnection.php(82): PDO->prepare()
#1 /path/httpdocs/vendor/illuminate/database/Connection.php(368): Doctrine\DBAL\Driver\PDOConnection->prepare()
#2 /path/httpdocs/vendor/illuminate/database/Connection.php(705): Illuminate\Database\Connection->Illuminate\Database\{closure}()
#3 /path/httpdocs/vendor/illuminate/database/Connection.php(672): Illuminate\Database\Connection->runQueryCallback()
#4 /path/httpdocs/vendor/illuminate/database/Connection.php(376): Illuminate\Database\Connection->run()
#5 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2414): Illuminate\Database\Connection->select()
#6 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2402): Illuminate\Database\Query\Builder->runSelect()
#7 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2936): Illuminate\Database\Query\Builder->Illuminate\Database\Query\{closure}()
#8 /path/httpdocs/vendor/illuminate/database/Query/Builder.php(2403): Illuminate\Database\Query\Builder->onceWithColumns()
#9 /path/httpdocs/vendor/illuminate/database/Eloquent/Builder.php(625): Illuminate\Database\Query\Builder->get()
#10 /path/httpdocs/vendor/illuminate/database/Eloquent/Builder.php(609): Illuminate\Database\Eloquent\Builder->getModels()
#11 /path/httpdocs/vendor/flarum/core/src/Search/AbstractSearcher.php(68): Illuminate\Database\Eloquent\Builder->get()
#12 /path/httpdocs/vendor/flarum/core/src/Api/Controller/ListDiscussionsController.php(100): Flarum\Search\AbstractSearcher->search()
#13 /path/httpdocs/vendor/flarum/core/src/Api/Controller/AbstractSerializeController.php(115): Flarum\Api\Controller\ListDiscussionsController->data()
#14 /path/httpdocs/vendor/flarum/core/src/Http/RouteHandlerFactory.php(41): Flarum\Api\Controller\AbstractSerializeController->handle()
#15 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ExecuteRoute.php(27): Flarum\Http\RouteHandlerFactory->Flarum\Http\{closure}()
#16 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ExecuteRoute->process()
#17 /path/httpdocs/vendor/kilowhat/flarum-ext-audit-pro/src/Middlewares/ExtendSetPermissionController.php(19): Laminas\Stratigility\Next->handle()
#18 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Kilowhat\Audit\Middlewares\ExtendSetPermissionController->process()
#19 /path/httpdocs/vendor/kilowhat/flarum-ext-audit-pro/src/Middlewares/SetLoggerActor.php(28): Laminas\Stratigility\Next->handle()
#20 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Kilowhat\Audit\Middlewares\SetLoggerActor->process()
#21 /path/httpdocs/vendor/flarum/core/src/Api/Middleware/ThrottleApi.php(33): Laminas\Stratigility\Next->handle()
#22 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Api\Middleware\ThrottleApi->process()
#23 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/CheckCsrfToken.php(36): Laminas\Stratigility\Next->handle()
#24 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\CheckCsrfToken->process()
#25 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ResolveRoute.php(67): Laminas\Stratigility\Next->handle()
#26 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ResolveRoute->process()
#27 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/SetLocale.php(51): Laminas\Stratigility\Next->handle()
#28 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\SetLocale->process()
#29 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/AuthenticateWithHeader.php(58): Laminas\Stratigility\Next->handle()
#30 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\AuthenticateWithHeader->process()
#31 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/AuthenticateWithSession.php(31): Laminas\Stratigility\Next->handle()
#32 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\AuthenticateWithSession->process()
#33 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/RememberFromCookie.php(52): Laminas\Stratigility\Next->handle()
#34 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\RememberFromCookie->process()
#35 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/StartSession.php(61): Laminas\Stratigility\Next->handle()
#36 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\StartSession->process()
#37 /path/httpdocs/vendor/flarum/core/src/Api/Middleware/FakeHttpMethods.php(29): Laminas\Stratigility\Next->handle()
#38 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Api\Middleware\FakeHttpMethods->process()
#39 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ParseJsonBody.php(28): Laminas\Stratigility\Next->handle()
#40 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ParseJsonBody->process()
#41 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/HandleErrors.php(57): Laminas\Stratigility\Next->handle()
#42 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\HandleErrors->process()
#43 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/InjectActorReference.php(25): Laminas\Stratigility\Next->handle()
#44 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\InjectActorReference->process()
#45 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(76): Laminas\Stratigility\Next->handle()
#46 /path/httpdocs/vendor/middlewares/request-handler/src/RequestHandler.php(84): Laminas\Stratigility\MiddlewarePipe->process()
#47 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\RequestHandler->process()
#48 /path/httpdocs/vendor/middlewares/base-path-router/src/BasePathRouter.php(101): Laminas\Stratigility\Next->handle()
#49 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\BasePathRouter->process()
#50 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Middleware/OriginalMessages.php(36): Laminas\Stratigility\Next->handle()
#51 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Laminas\Stratigility\Middleware\OriginalMessages->process()
#52 /path/httpdocs/vendor/middlewares/base-path/src/BasePath.php(73): Laminas\Stratigility\Next->handle()
#53 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Middlewares\BasePath->process()
#54 /path/httpdocs/vendor/flarum/core/src/Http/Middleware/ProcessIp.php(24): Laminas\Stratigility\Next->handle()
#55 /path/httpdocs/vendor/laminas/laminas-stratigility/src/Next.php(51): Flarum\Http\Middleware\ProcessIp->process()
#56 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(76): Laminas\Stratigility\Next->handle()
#57 /path/httpdocs/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(65): Laminas\Stratigility\MiddlewarePipe->process()
#58 /path/httpdocs/vendor/laminas/laminas-httphandlerrunner/src/RequestHandlerRunner.php(96): Laminas\Stratigility\MiddlewarePipe->handle()
#59 /path/httpdocs/vendor/flarum/core/src/Http/Server.php(44): Laminas\HttpHandlerRunner\RequestHandlerRunner->run()
#60 /path/httpdocs/public/index.php(26): Flarum\Http\Server->listen()
#61 {main}

clarkwinkelmann

GreXXL it seems to be an issue caused by the use of a database table prefix. I'll look into this right away.

CyberGene If I search for "Chinese", I get results from other discussions and posts but not this one

Yep, that's odd. I haven't tested Algolia yet so not sure what's happening. I'll try replicating with Meilisearch this afternoon and see if it gives me any clue.

I'll need to do a proper and complete Algolia test at some point but haven't had the time yet.

CyberGene I exceeded the quota

Yep, every post (including first post of discussions), every discussion and every user will most likely count as one object. There's currently no way to exclude a specific resource but I could imagine adding an option to exclude users from Scout if this helps reduce service cost and isn't absolutely needed.

CyberGene

I installed it, imported the posts into the index and the search even works great, matching partial search terms...

However I am getting 500 while browsing the forum 😕

CyberGene

OK, I got it, in the Flarum logs there's Algolia message saying I exceeded the quota (10k objects and we have 11k posts 😀)... So, I upgraded to the $1 a unit tier, let's see how much it will cost, my understanding is $1 for each additional 1000 objects.

Hari

They got opensource version too 🧐 I will try this

https://docs.meilisearch.com/learn/cookbooks/digitalocean_droplet.html

CyberGene Algolia is expensive, go through this old discussion luceos

CyberGene

Seems to be working well so far, thanks a lot! Much better than the embedded SQL search.

I see some odd situations though. For instance I created a post where I have a sentence containing "Chinese manufactured". If I search for "Chinese", I get results from other discussions and posts but not this one. If I search for "Chinese manufactured", then it's found. A bit odd, why would that happen?

clarkwinkelmann

Version 0.1.2

Fixed support for using Flarum with a database prefix
Fixed posts not syncing to index when created through a discussion or a reply
Fixed old content being synced when a post is edited

I recommend running the scout:import-all command to fix all existing broken records.

This update changes the way post content is indexed. Previously the markdown/bbcode version of the post from edit mode would be sent to the index. Now it's the rendered HTML version except it's stripped of any HTML. This should prevent hidden information from being leaked via search (example: old username if a mentioned user changed their username, content hidden with the hide bbcode/reply/likes extensions). But it also means image and link URLs are now no longer indexed. I will try to add an option to get those URLs indexed again in a next version.

CyberGene I was not able to replicate your issue on Meilisearch, however I found out the synchronization wasn't working correctly. So maybe your new post wasn't indexed yet.

If the problem happens again, can you try performing the search directly in Algolia's dashboard? To see if any result exists in the search index? Meilisearch has a nice built-in web interface that allows inspecting the current state of the index and I believe Algolia has something similar.

CyberGene

clarkwinkelmann thanks, I installed the update and reimported the index. Seems like the search issue I had is fixed now, so maybe the problem was with what you described as possible reason. I will let you know if there are some other issues. So far all looks pretty good, thanks a lot! 🤝

CyberGene

Hari Thanks, I may check that later after I see how much Algolia will cost me this month.

CyberGene

Hari what's this digital ocean service, is it verified? I'm trying to register myself and can't go through the initial payment validation. I tried every possible payment method: Apple Pay, PayPal, direct credit card with all my cards and nothing goes through, they are all rejected for some reason. I'm giving up...

Hari

CyberGene as far i know you need not to pay any money to set-up a $5 droplet 🤔 they do accept debit cards too. And you get $100 free credit as a sign-up bonus (not sure about this as they advertise this)

CyberGene

Never mind, registered with Revolut, maybe my bank is having some problems currently.

CyberGene

clarkwinkelmann If you are regularly running composer update on all your dependencies, you should add an explicit requirement for the Meilisearch SDK in your composer.json because the extension requires * which might jump to a newer Meilisearch SDK version as soon as it comes out.

Can you tell me what exact stuff should I put in the composer.json?

I switched to Meilisearch in DigitalOcean, as explained by the link Hari posted. I imported the indexes, all works. I guess I will use it instead of Algolia because it's cheaper. Thanks Hari

Hari

CyberGene let's discuss here about Meilisearch

Was it good? How much RAM it's occupying? I am thinking to run in a docker on my 4gb RAM droplet

What are the installation steps?

tohsakarat

Hello, I use the extension with meili search and it works well, thank you!
However I find that the results that match content is prior to those match title. How shall I change the priority?
Thank you!

clarkwinkelmann

tohsakarat unfortunately it's a consequence of the current implementation, so can't be changed.

I'm not sure what the best way to calculate the sort order is. The problem is there are 2 indexes, one for discussion data (title) and one for post data (content). And I can't get the Meilisearch matching score out of Scout, only the order of the results. So merging the 2 in a way that makes sense is quite challenging. I guess it will require some specific code to go deeper level into Meilisearch API or modify the logic to use a single index.

clarkwinkelmann

Version 0.2.0 - TNTSearch

Following a suggestion by @010101 I have added TNTSearch support to the extension.

Existing users don't need to install this update if they are not planning to try TNTSearch.

To update:

composer require clarkwinkelmann/flarum-ext-scout

Additionally, to use TNTSearch you need to install an additional package. This package requires the sqlite PHP extension on the server!

composer require teamtnt/laravel-scout-tntsearch-driver

See the README on GitHub for more information https://github.com/clarkwinkelmann/flarum-ext-scout#tntsearch

If you switch driver, you need to rebuild the indexes with the command!

From my tests, TNTSearch seems an incredibly poor search engine. Their documentation is very lacking and the few options/syntaxes that are described don't seem to work. Maybe it can be improved slightly by playing with the settings, but I'm not sure it brings much improvement over Flarum native search.

This extension is still very experimental and designed to test alternate search engines with the goal of maybe integrating some of them natively in Flarum in the future.

fakruzaruret

clarkwinkelmann Why you prefered the TNT instead of Meilisearch?

010101

clarkwinkelmann That’s disappointing that TNT search ended up not being all that great. Maybe the devs for that will continue to improve it.

clarkwinkelmann

fakruzaruret I'm not sure what you mean.

The 3 drivers I've tested so far are Flarum native search, Meilisearch and TNTSearch. I have still not tested Algolia myself despite the extension supporting it.

I didn't see much of a difference between Flarum native and TNTSearch. I tried TNTSearch negative keyword syntax and tried searching sentences but it doesn't seem to work. It just works with simple keywords, much like Flarum native search.

Meilisearch is still my favorite solution so far, it works great and is incredible at searching for partial words and wrong spellings. I have not had an opportunity to use it in production yet, but I'm hoping to install it on one of my forums later this year.

CyberGene

I've been using Meilisearch through this extension for some time now and I'm very happy with it. The only thing is I have to pay €7 a month for the DigitalOcean droplet and while that's not much, I'm still not sure if it's worth it over the Flarum native search. I'll have to check with my users about how often they use the search functionality but it seems they don't care a lot and don't use it too much, so I may switch to Flarum native again unless I find cheaper Meilisearch solution since my current needs are just too tiny to justify even €7 a month.

clarkwinkelmann

CyberGene where are you hosting your forum? You might be able to place Flarum and Meilisearch on the same server to save on cost.

Meilisearch seem to be working on a cloud hosting, but there's just a waitlist and no mention of the pricing. I doubt it would be affordable for this kind of use case but we never know.

It's true that if few people use the search bar, switching driver might not make a lot of sense. One place where people might not immediately realize is using the search are mentions. The auto-complete relies on the same user search endpoint. But user search is also not dramatically improved over the native search since it's just usernames that are indexed anyway.