The language is set correctly (German) and I have already deleted the index "flarum_forum" completely and created it again.
It seems that it does not find results when my search string is longer than 10 characters.
Blomstra Search, an elasticsearch fulltext search extension
maisen20 I don't see this issue in production: https://forum.mutluanneleriz.com/?q=hastane%20%C3%A7antas%C4%B1n%C4%B1%20haz%C4%B1rlamaya%20eksik%20hep%20%C3%A7%C4%B1k%C4%B1yor%20%C3%A7%C3%BCnk%C3%BC%20birde
Can you try identifying what is sent through the Flarum api?
- Edited
When i check the console i can see the request url is:
/api/blomstra/search/discussions?filter[q]=filtervorlagen&page[limit]=3&include=mostRelevantPost
I can't see errors in the logs and the result is:
{"links":{"first":"https:\/\/community.domain.com\/api\/blomstra\/search\/discussions?filter%5Bq%5D=filtervorlagen\u0026page%5Blimit%5D=20\u0026include=mostRelevantPost\u0026type=discussions"},"data":[]}
To inform everyone:
I figured out what the reason for the limitation to 10 characters is.
In the file "/vendor/blomstra/search/src/Commands/BuildCommand.php" there are the values for 'index.max_ngram_diff' and 'max_gram' which tell ES's tokenizer how long the token can be. These values are set to 10. I assume that this is the case for all of you.
This means words containing more than 10 characters will not be tokenized and therefore these words/posts will not be found.
5 months later
composer require blomstra/search:"*"
l want install this extension but write a error words
luceos Thank you. Worked 
luceos but not understand. What is?
Elastic Endpoint
Username
Password
Index
and not have Analyzer language : Azerbaijan
and how to fixed size searchbox?
- Edited
Mirzaagha Elastic Endpoint
Username
Password
Index
these are settings your elasticsearch instance has
Mirzaagha Azerbaijan
I don't know, your elastic instance probably needs to be configured with it. We currently have no flexible way to set this as a value, but you could modify the setting inside your database to the given analyzer, it might work. Otherwise a PR with the changes needed to make the field flexible would be appreciated.
2 months later
may be orama tool is usable for text searches. I would like to share as an option
https://docs.askorama.ai/
2 months later
- Edited
@luceos I cannot get this extension work. After running the php flarum blomstra:search:index --recreate command, it seems to index everything ("Pushed a total of 5309387 into the index."), but the search results are empty. There are no errors.
I checked the indexes, and it looks like nothing is saved though, because the index takes up very little space and I tested it on 5 million posts.
~$ curl http://localhost:9200/_cat/indices?pretty=true
yellow open localhost 9cVCxB2gQ6W_ZTvCMOWjvg 1 1 0 0 249b 249b 249bWhen I run the php flarum queue:work command, there are these errors in the logs:
[2024-06-22 09:44:03] flarum.ERROR: Blomstra\Search\Exceptions\SeedingException: Failed to seed: Index 0 out of bounds for length 0 in /var/www/html/vendor/blomstra/search/src/Jobs/SavingJob.php:53
Stack trace:
#0 /var/www/html/vendor/illuminate/container/BoundMethod.php(36): Blomstra\Search\Jobs\SavingJob->handle(Object(Elasticsearch\Client))
#1 /var/www/html/vendor/illuminate/container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#2 /var/www/html/vendor/illuminate/container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#3 /var/www/html/vendor/illuminate/container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Container\Container), Array, Object(Closure))
#4 /var/www/html/vendor/illuminate/container/Container.php(653): Illuminate\Container\BoundMethod::call(Object(Illuminate\Container\Container), Array, Array, NULL)
#5 /var/www/html/vendor/illuminate/bus/Dispatcher.php(128): Illuminate\Container\Container->call(Array)
#6 /var/www/html/vendor/illuminate/pipeline/Pipeline.php(128): Illuminate\Bus\Dispatcher->Illuminate\Bus\{closure}(Object(Blomstra\Search\Jobs\SavingJob))
#7 /var/www/html/vendor/illuminate/pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Blomstra\Search\Jobs\SavingJob))
#8 /var/www/html/vendor/illuminate/bus/Dispatcher.php(132): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#9 /var/www/html/vendor/illuminate/queue/CallQueuedHandler.php(119): Illuminate\Bus\Dispatcher->dispatchNow(Object(Blomstra\Search\Jobs\SavingJob), false)
#10 /var/www/html/vendor/illuminate/pipeline/Pipeline.php(128): Illuminate\Queue\CallQueuedHandler->Illuminate\Queue\{closure}(Object(Blomstra\Search\Jobs\SavingJob))
#11 /var/www/html/vendor/illuminate/pipeline/Pipeline.php(103): Illuminate\Pipeline\Pipeline->Illuminate\Pipeline\{closure}(Object(Blomstra\Search\Jobs\SavingJob))
#12 /var/www/html/vendor/illuminate/queue/CallQueuedHandler.php(118): Illuminate\Pipeline\Pipeline->then(Object(Closure))
#13 /var/www/html/vendor/illuminate/queue/CallQueuedHandler.php(70): Illuminate\Queue\CallQueuedHandler->dispatchThroughMiddleware(Object(Illuminate\Queue\Jobs\RedisJob), Object(Blomstra\Search\Jobs\SavingJob))
#14 /var/www/html/vendor/illuminate/queue/Jobs/Job.php(98): Illuminate\Queue\CallQueuedHandler->call(Object(Illuminate\Queue\Jobs\RedisJob), Array)
#15 /var/www/html/vendor/illuminate/queue/Worker.php(428): Illuminate\Queue\Jobs\Job->fire()
#16 /var/www/html/vendor/illuminate/queue/Worker.php(378): Illuminate\Queue\Worker->process(NULL, Object(Illuminate\Queue\Jobs\RedisJob), Object(Illuminate\Queue\WorkerOptions))
#17 /var/www/html/vendor/illuminate/queue/Worker.php(172): Illuminate\Queue\Worker->runJob(Object(Illuminate\Queue\Jobs\RedisJob), NULL, Object(Illuminate\Queue\WorkerOptions))
#18 /var/www/html/vendor/illuminate/queue/Console/WorkCommand.php(116): Illuminate\Queue\Worker->daemon(NULL, 'default', Object(Illuminate\Queue\WorkerOptions))
#19 /var/www/html/vendor/illuminate/queue/Console/WorkCommand.php(100): Illuminate\Queue\Console\WorkCommand->runWorker(NULL, 'default')
#20 /var/www/html/vendor/illuminate/container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#21 /var/www/html/vendor/illuminate/container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#22 /var/www/html/vendor/illuminate/container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure(Object(Closure))
#23 /var/www/html/vendor/illuminate/container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod(Object(Illuminate\Container\Container), Array, Object(Closure))
#24 /var/www/html/vendor/illuminate/container/Container.php(653): Illuminate\Container\BoundMethod::call(Object(Illuminate\Container\Container), Array, Array, NULL)
#25 /var/www/html/vendor/illuminate/console/Command.php(136): Illuminate\Container\Container->call(Array)
#26 /var/www/html/vendor/symfony/console/Command/Command.php(298): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#27 /var/www/html/vendor/illuminate/console/Command.php(120): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Illuminate\Console\OutputStyle))
#28 /var/www/html/vendor/symfony/console/Application.php(1058): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#29 /var/www/html/vendor/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand(Object(Flarum\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#30 /var/www/html/vendor/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#31 /var/www/html/vendor/flarum/core/src/Console/Server.php(42): Symfony\Component\Console\Application->run()
#32 /var/www/html/flarum(24): Flarum\Console\Server->listen()
#33 {main} Elasticsearch version: 8.13.4
- Edited
luceos like how is your queue set up
I'm using blomstra/flarum-redis for queue.
Flarum info:
I'm testing this extension on a local docker environment currently.
Flarum core: 1.8.5
PHP version: 8.3.8
MySQL version: 10.5.23-MariaDB-1:10.5.23+maria~ubu2004
Loaded extensions: Core, date, libxml, openssl, pcre, sqlite3, zlib, ctype, curl, dom, fileinfo, filter, hash, iconv, json, mbstring, SPL, session, PDO, pdo_sqlite, standard, posix, random, readline, Reflection, Phar, SimpleXML, tokenizer, xml, xmlreader, xmlwriter, mysqlnd, gd, pdo_mysql, sodium, zip
+-----------------------------------+---------------------------------------+--------+
| Flarum Extensions | | |
+-----------------------------------+---------------------------------------+--------+
| ID | Version | Commit |
+-----------------------------------+---------------------------------------+--------+
| flarum-flags | v1.8.0 | |
| flarum-tags | v1.8.0 | |
| flarum-approval | v1.8.1 | |
| flarum-lock | v1.8.0 | |
| blomstra-gdpr | 0.1.0-beta.20 | |
| flarum-mentions | v1.8.3 | |
| flarum-subscriptions | v1.8.0 | |
| v17development-support | v2.7.2 | |
| flarum-suspend | v1.8.1 | |
| fof-follow-tags | 1.2.2 | |
| fof-byobu | 1.3.6 | |
| acpl-mobile-tab | 1.4.3 | |
| fof-user-directory | 1.3.3 | |
| acpl-communities | dev-master | |
| flarum-markdown | v1.8.0 | |
| fof-masquerade | 2.1.4 | |
| fof-pages | 1.0.8 | |
| fof-best-answer | 1.4.4 | |
| fof-oauth | 1.6.9 | |
| fof-drafts | 1.2.10 | |
| acpl-gamification | dev-master | |
| fof-nightmode | 1.5.3 | |
| flarumite-simple-discussion-views | 1.2.2 | |
| v17development-seo | v1.8.1 | |
| the-turk-diff | dev-fix-parse-and-render-missing-args | |
| sycho-move-posts | v0.1.7 | |
| ianm-follow-users | 1.4.6 | |
| fof-username-request | 1.2.4 | |
| fof-user-bio | 1.3.2 | |
| fof-upload | 1.5.5 | |
| fof-terms | 1.3.0 | |
| fof-socialprofile | 1.1.6 | |
| fof-polls | 2.2.2 | |
| fof-merge-discussions | 1.4.1 | |
| fof-ignore-users | 1.2.1 | |
| fof-geoip | 1.4.3 | |
| fof-formatting | 1.0.3 | |
| fof-anti-spam | 1.1.2 | |
| flarum-statistics | v1.8.0 | |
| flarum-lang-polish | dev-master | |
| flarum-lang-english | v1.8.0 | |
| flarum-emoji | v1.8.0 | |
| flarum-bbcode | v1.8.0 | |
| clarkwinkelmann-author-change | 1.0.3 | |
| askvortsov-rich-text | v2.1.7 | |
| askvortsov-pwa | v3.4.1 | |
| askvortsov-moderator-warnings | v0.6.3 | |
| acpl-moderation-tools | dev-master | |
| acpl-forum | dev-master | |
| acpl-follow-communities | dev-master | |
| acpl-ads | dev-master | |
+-----------------------------------+---------------------------------------+--------+
Base URL: http://facp.localhost
Installation path: /var/www/html
Queue driver: redis
Session driver: redis (Code override. Configured to file)
Mail driver: log
Debug mode: ONThere are custom extensions here, but I tested with them disabled also.
2 months later
Unfortunately, my posts/discussions have not been synchronized with the elasticsearch for weeks or months. Reindex works via manual command per cli. I have been searching for the problem for about 6 hours now, but apparently I don't quite understand it.
Please note the following:
- Redis works without problems
- Other jobs are processed without any issue by the system (e.g. Flarum\Mentions\Job\SendMentionsNotificationsJob)
- Elasticsearch works
- Manual recreation of the elasticsearch index works
The problem is on prod and dev environment.
Flarum core: 1.8.5
PHP version: 8.2.22
MySQL version: 10.6.19-MariaDB-ubu2004
Loaded extensions: Core, date, libxml, openssl, pcre, sqlite3, zlib, ctype, curl, dom, fileinfo, filter, hash, iconv, json, mbstring, SPL, session, PDO, pdo_sqlite, standard, posix, random, readline, Reflection, Phar, SimpleXML, tokenizer, xml, xmlreader, xmlwriter, mysqlnd, bcmath, calendar, exif, gd, gettext, gmp, imagick, imap, intl, memcached, mysqli, pcntl, pdo_mysql, redis, sodium, zip
+--------------------------------------+----------+--------+
| Flarum Extensions | | |
+--------------------------------------+----------+--------+
| ID | Version | Commit |
+--------------------------------------+----------+--------+
| flarum-flags | v1.8.0 | |
| flarum-tags | v1.8.0 | |
| flarum-approval | v1.8.1 | |
| flarum-suspend | v1.8.1 | |
| flarum-mentions | v1.8.3 | |
| flarum-subscriptions | v1.8.0 | |
| flarum-markdown | v1.8.0 | |
| flarum-sticky | v1.8.0 | |
| flamarkt-backoffice | 0.1.3 | |
| fof-byobu | 1.3.6 | |
| fof-user-directory | 1.3.3 | |
| fof-follow-tags | 1.2.2 | |
| therealsujitk-gifs | v4.1.1 | |
| the-turk-stickiest | 3.0.1 | |
| maicol07-sso | 1.11.5 | |
| ianm-synopsis | 1.3.2 | |
| fof-user-bio | 1.3.2 | |
| fof-upload | 1.5.4 | |
| fof-subscribed | 1.1.4 | |
| fof-links | 1.2.3 | |
| fof-linguist | 1.1.2 | |
| fof-impersonate | 1.1.1 | |
| fof-ignore-users | 1.2.1 | |
| fof-drafts | 1.2.10 | |
| fof-default-user-preferences | 1.2.1 | |
| fof-best-answer | 1.4.2 | |
| flarum-statistics | v1.8.0 | |
| flarum-nicknames | v1.8.0 | |
| flarum-lock | v1.8.0 | |
| flarum-likes | v1.8.0 | |
| flarum-lang-german | 1.10.0 | |
| flarum-emoji | v1.8.0 | |
| flarum-bbcode | v1.8.0 | |
| flamarkt-matomo | 0.1.0 | |
| datlechin-link-preview | v1.4.0 | |
| datlechin-landing-page | 0.1.0 | |
| clarkwinkelmann-post-stream-search | 1.1.0 | |
| clarkwinkelmann-first-post-approval | 1.0.1 | |
| clarkwinkelmann-discussion-bookmarks | 2.0.1 | |
| clarkwinkelmann-collapsible-posts | 1.0.1 | |
| blomstra-search | 0.1.19 | |
| askvortsov-rich-text | v2.1.7 | |
| askvortsov-moderator-warnings | v0.6.3 | |
| askvortsov-markdown-tables | v1.2.1 | |
+--------------------------------------+----------+--------+
Base URL: ***
Installation path: ***
Queue driver: redis
Session driver: redis (Code override. Configured to file)
Scheduler status: aktiv
Mail driver: smtp
Debug mode: off11 days later
As far as I can tell, changes to posts are not being updated in the Elasticsearch database. Or am I mistaken?
a month later
I can't get past the "No alive nodes in your cluser" message. Elasticsearch is running, there is a node, there is an index. I can get the list of indexes from it by calling curl blah blah /_aliases. I have entered the host/user/password/index correctly in the flarum admin, but I still get "No alive nodes in your cluster" when I try running php flarum blomstra:search:index
I wonder ... does it have something to do with SSL? I noticed most examples above are http://localhost:9200. My elasticsearch seems to be running on HTTPS. Is that a problem?
luceos It does.
When I query the service using the command from the ES docs, it works:
curl --cacert $ES_HOME/config/certs/http_ca.crt -u elastic:$ELASTIC_PASSWORD https://localhost:9200
yields
{
"name" : "node-1",
"cluster_name" : "cluster-1",
"cluster_uuid" : "jwMl-ycHSCiPyEwQKnAlyw",
"version" : {
"number" : "8.15.2",
"build_flavor" : "default",
"build_type" : "tar",
"build_hash" : "98adf7bf6bb69b66ab95b761c9e5aadb0bb059a3",
"build_date" : "2024-09-19T10:06:03.564235954Z",
"build_snapshot" : false,
"lucene_version" : "9.11.1",
"minimum_wire_compatibility_version" : "7.17.0",
"minimum_index_compatibility_version" : "7.0.0"
},
"tagline" : "You Know, for Search"
}
I have all the same settings set in flarum:
host: https://localhost:9200
username: elastic
password: <$ELASTIC_PASSWORD>
index: flarum-search
and yet when I run php flarum blomstra:search:index i get
In StaticNoPingConnectionPool.php line 64:
No alive nodes found in your cluster