• SupportSolved

  • why flarum search is bad and how to improve with a new extension?

Hari Is there any reason why you haven't considered blomstra/search (https://discuss.flarum.org/d/30404-blomstra-search-an-elasticsearch-fulltext-search-extension)? I honestly don't have that much experience with these search related issues in Flarum (I know of its existence, but it's not something that is affecting my use cases) so I can't help you as much as I'd like to, but Elasticsearch is a very well known enterprise wide option, in fact I have some contact with that search option because I manage several Magento instances, so its performance in Flarum should be pretty good.

      Darkle it's not working for non Latin languages like Telugu, Tamil, Hindi.

      @clarkwinkelmann could you please go through this Hari and tell a solution

          does the develop team want to improve this function? do they have a plan?

          Is the issue here that search is limited to at least three characters? Because if that's the case we could easily make that configurable, even an extension could do that.

            Hari If i need to migrate i would chose wordpress over discourse.

            Going little off topic, but don't go on WP route. All their forum softwares/extensions are slow and sluggish. WP is great blogging software, but only that. I made that mistake by switching from SMF and it nearly killed my otherwise very active and numerous community. Luckily, Flarum saved us.

              luceos If I remember correctly it would be changing the table from MyIASM to InnoDB, which can take a very, very long time on larger tables/databases and then it would also require re-indexing the fulltext search (which would also take a long time)

              • Hari replied to this.

                  Hari Have you tried it if it works? I think it would be a good thing to document in the resources tag.

                    SychO currently flarum using InnoDB? Or flarum test environment using innoDB?

                        Hari never mind actually, Flarum has used InnoDB by default for far longer than that. I just investigated our codebase and there is a bug preventing languages like Telugu from being searched, I've posted a fix here: flarum/framework3385

                            SychO thanks for the quick fix. ♥️♥️♥️♥️♥️ Like Shasha said this gonna be the biggest PR in flarum's history for sure.

                            Please give me the test environment URL I would like to test, as I can see you have mentioned that you did these changes locally, could you deploy it somewhere.

                            I will take a few songs lyrics and perform a search

                            Telugu

                            ఒకటే జననం ఒకటే మరణం
                            ఒకటే గమనం ఒకటే గమ్యం
                            గెలుపు పొందె వరకూ అలుపు లేదు మనకు..
                            బతుకు అంటె గెలుపూ గెలుపుకొరకె బ్రతుకు..
                            కష్టాలు రానీ కన్నీళ్ళు రానీ
                            ఏమైన గానీ ఎదురేది రానీ
                            ఓడిపోవద్దు రాజీపడొద్దు
                            నిద్ర నీకొద్దు నీకేది హద్దు

                            Hindi

                            किसी को सपना लगे तू … किसी को बहती हवा किसी को बस बातों में, करे पल में यहाँ वहाँ किसी के सौ झूठ सुन ले, किसी का सच्च भी गुनाह किसी का बस यादों में, करे हलचल सारा जहां टी यामो मी टी यमो तू छाँव है, तू धुप है टी यामो मी टी यमो तेरे हजारों रूप हैं

                            Kannada

                            (ಗ) ನಿನ್ನ ಜೊತೆ ನನ್ನ ಕಥೆ
                            ಒಂದೊಂದು ಸಾಲು ಜೀವಿಸಿದೆ
                            ನನ್ನ ಜೊತೆ ನಿನ್ನ ಕಥೆ
                            ಬೇರೊಂದು ಲೋಕ ಸೃಷ್ಟಿಸಿದೆ

                            ಎಂದು ಹೀಗೆ ಆಗೇ ಇಲ್ಲ
                            ಏನು ಇದರ ಸೂಚನೆ
                            ನೂರು ವಿಷತ ಇದ್ದರೂನು
                            ನಿನ್ನದೊಂದೇ ಯೋಚನೆ

                            Malyalam

                            ഇന്‍റര്‍നെറ്റിനെ ആകമാനം പിടിച്ചുലച്ച ഗാനമാണ് വെളിപാടിന്‍റെ പുസ്തകം എന്ന ചിത്രത്തലെ എന്‍റമ്മേടെ ജിമിക്കി കമ്മല്‍ എന്ന് തുടങ്ങുന്ന ഗാനം. പാട്ടും അതിനൊത്ത താളച്ചുവടുകളും കൊണ്ട് കേരളം മാത്രമല്ല, മറ്റ് സംസ്ഥാനങ്ങളിലും ഈ പാട്ട് അലകളുണ്ടാക്കി. പാട്ടിന്‍റെ മുഴുവൻ വരികൾ ഇതാ..

                            Tamil

                            நியூயார்க் நகரம்
                            உறங்கும் நேரம் தனிமை
                            அடர்ந்தது பனியும் படர்ந்தது
                            கப்பல் இறங்கியே காற்றும்
                            கரையில் நடந்தது

                            நான்கு கண்ணாடி
                            சுவர்களுக்குள்ளே நானும்
                            மெழுகுவர்த்தியும் தனிமை
                            தனிமையோ கொடுமை
                            கொடுமையோ

                            Sinhala

                            ෆ්ලාරම් සඳහා සිංහල භාෂා ඇසුරුම

                            https://discuss.flarum.org/d/27076-sinhala-language-pack

                                Hari the pr is merged, not sure but probably you can install Flarum dev master locally into your pc?

                                • Hari replied to this.

                                    Justoverclock how to install dev-master, what is the command? is the below command correct?

                                    composer create-project flarum/flarum dev master

                                    Edit:

                                    git clone git@github.com:flarum/flarum.git <target directory>
cd <target directory>

                                    https://discuss.flarum.org/d/9829-installing-flarum-dev-master

                                    looks like not easy. can i try this? https://demo.flarum.site/

                                        Hari The dev-master demo Nightly (https://nightly.flarum.site/) is under 6f72c7f not sure if it is updated or not.
                                        If you prefer local, I think the easiest thing to do would be:

                                        composer create-project flarum/flarum:"@"dev" Sorry, forget it, it's not like that to install it locally.

                                          YAYYYYYY! search is working 🧨✨🎇🎉 The reason I am mentioning these many languages is I want flarum to consider every langue equally so the search will work in any scenario

                                          ✅ Telugu is working: ఒకటే జననం

                                          https://nightly.flarum.site/?q=ఒకటే%20జననం

                                          ❌ Hindi is not working: किसी को

                                          https://nightly.flarum.site/?q=किसी%20को

                                          ❌ Kannada is not working: ಸಾಲು ಜೀವಿಸಿದೆ

                                          https://nightly.flarum.site/?q=ಸಾಲು%20ಜೀವಿಸಿದೆ

                                          ❌ Malayalam is not working: പിടിച്ചുലച്ച ഗാനമാണ്

                                          https://nightly.flarum.site/?q=പിടിച്ചുലച്ച%20ഗാനമാണ്

                                          ❌ Tamil is not working : நியூயார்க்

                                          https://nightly.flarum.site/?q=நியூயார்க்

                                          ❌ Sinhala is not working: සිංහල භාෂා

                                          https://nightly.flarum.site/?q=සිංහල%20භාෂා

                                          i request to support all the possible languages from this Unicode Scripts

                                          https://wpastra.com/docs/complete-list-wordpress-locale-codes/

                                          \p{Common}
\p{Arabic}
\p{Armenian}
\p{Bengali}
\p{Bopomofo}
\p{Braille}
\p{Buhid}
\p{Canadian_Aboriginal}
\p{Cherokee}
\p{Cyrillic}
\p{Devanagari}
\p{Ethiopic}
\p{Georgian}
\p{Greek}
\p{Gujarati}
\p{Gurmukhi}
\p{Han}
\p{Hangul}
\p{Hanunoo}
\p{Hebrew}
\p{Hiragana}
\p{Inherited}
\p{Kannada}
\p{Katakana}
\p{Khmer}
\p{Lao}
\p{Latin}
\p{Limbu}
\p{Malayalam}
\p{Mongolian}
\p{Myanmar}
\p{Ogham}
\p{Oriya}
\p{Runic}
\p{Sinhala}
\p{Syriac}
\p{Tagalog}
\p{Tagbanwa}
\p{TaiLe}
\p{Tamil}
\p{Telugu} ✅
\p{Thaana}
\p{Thai}
\p{Tibetan}
\p{Yi}

                                            Hari I tested all of those locally and they actually work, I'm not entirely sure if nightly is updating since we switched to a monorepo.

                                            here's a good way of figuring out if certain language characters are not filtered out. If a character is highlighted then it's a problem: https://regex101.com/r/qesbFx/1 I can already see there is still an issue with Malayalam lang, not sure yet how to escape those.