Improving search for Chinese characters

Littlegolden

fromsly I found that the Chinese search for this forum is nice: https://freeerror.org/

Littlegolden Maybe I should try to change the minimum word length limit of search keywords to 1?

luceos This is exactly what Clark and I were hoping was causing all these troubles.

I found that the website @fromsly said can do search only use one character.

fromsly

@Littlegolden Gentlemen, this is a post discussing the difference between Chinese search and English search：https://www.v2ex.com/t/203939

The pain points and difficulties of Chinese search are also discussed in depth.(并且深度讨论了中文搜索的痛点难点。)

samtto

After flarum is used in the field of production, many problems are solved quickly. However, when the data increases greatly, we are eager to find out the history by searching the title.

Littlegolden

luceos I changed if (query.length >= 1), then I did php flarum cache:clear , but nothing happened. After 10 hours I still can not search one character...

tankerkiller125

Littlegolden Did you also compile the JS after making the change (npm install in the js directory and then npm run build) if you didn't compile the JS after the change that may part of the reason the change isn't taking place.

Littlegolden

tankerkiller125 Looks something went wrong?

[root@VM_0_11_centos components]# npm install
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/unique-slug-bdaf2238/test/index.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/source-map-a60e38ce/dist/source-map.debug.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/plugin-transform-dotall-regex-b918722d/LICENSE'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/upath-f179734e/build/code/upath.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/plugin-transform-dotall-regex-b918722d/README.md'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/jquery-fc74c5ce/external/sizzle/dist/sizzle.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/webpack-e60c0cf6/lib/Compiler.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/moment-f7a07c1e/min/moment-with-locales.min.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/ajv-626cb654/dist/ajv.min.js'
npm ERR! code 128
npm ERR! Command failed: git clone --mirror -q git://github.com/tobscure/m.attrs.bidi.git /root/.npm/_cacache/tmp/git-clone-9cbdc1b7/.git
npm ERR! fatal: Can not reate a pilot directory for '/root/.npm/_cacache/tmp/git-clone-9cbdc1b7/.git' 
npm ERR! 

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-01-09T17_34_15_241Z-debug.log
[root@VM_0_11_centos components]# npm run build

> @flarum/core@ build /www/wwwroot/bbs.csur.fun/vendor/flarum/core/js
> webpack --mode production

sh: webpack: command not found
npm ERR! code ELIFECYCLE
npm ERR! syscall spawn
npm ERR! file sh
npm ERR! errno ENOENT
npm ERR! @flarum/core@ build: `webpack --mode production`
npm ERR! spawn ENOENT
npm ERR! 
npm ERR! Failed at the @flarum/core@ build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-01-09T17_35_06_386Z-debug.log

2020-01-09T17_34_15_241Z-debug.log
2020-01-09T17_35_06_386Z-debug.log

tankerkiller125

This looks like a potential permission error or disk space issue of some kind until npm install comes back clean npm run build won't work. Unfortunately I don't know exactly where/why NPM is getting hung up creating a temp directory/file.

luceos

root@VM_0_11_centos components

To me it seems you are running the commands in the wrong directory. Aren't you in one too deep? You should be in /www/wwwroot/bbs.csur.fun/vendor/flarum/core/js

cd /www/wwwroot/bbs.csur.fun/vendor/flarum/core/js
npm install
npm build
// to clear node modules massive amounts of data
rm -rf node_modules

Littlegolden

luceos I tried this time... still got the same error I think.

Full log file
- https://drive.google.com/open?id=1hbYI1FV-M2YpfJbyv1ohI8GWCkCF7DHv

[root@VM_0_11_centos ~]# cd /www/wwwroot/bbs.csur.fun/vendor/flarum/core/js
[root@VM_0_11_centos js]# npm install
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/has-values-a73b33ba/package.json'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/has-values-a73b33ba/README.md'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/helper-module-imports-5b65ee3f/LICENSE'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/plugin-proposal-async-generator-functions-b129a5c8/LICENSE'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/has-values-a73b33ba/LICENSE'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/helper-module-imports-5b65ee3f/README.md'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/has-values-a73b33ba/index.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/helper-split-export-declaration-74debc45/package.json'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/helper-split-export-declaration-74debc45/lib/index.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/generator-2d643897/lib/node/index.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/generator-2d643897/lib/node/parentheses.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/generator-2d643897/lib/node/whitespace.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/generator-2d643897/lib/printer.js'
npm WARN tar ENOENT: no such file or directory, open '/www/wwwroot/bbs.csur.fun/vendor/flarum/core/js/node_modules/.staging/@babel/generator-2d643897/lib/source-map.js'
npm ERR! code 128
npm ERR! Command failed: git clone --mirror -q git://github.com/tobscure/m.attrs.bidi.git /root/.npm/_cacache/tmp/git-clone-85a69dd5/.git
npm ERR! fatal: Can not create a pilot directory for '/root/.npm/_cacache/tmp/git-clone-85a69dd5/.git'
npm ERR! 

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-03-03T13_25_33_613Z-debug.log
[root@VM_0_11_centos js]# cd /root/.npm/_logs/

samtto

Littlegolden 以上clarkwinkelmann 的回复，我是小菜菜，改改皮皮都废气，靠你来抽点时间给个TEST CASES.
Eenlish: as calrkwinkelmann said,Pls Do Me A Favor to give the test cases.TKS.
Thanks to all.

samtto

Looking forward to the official update in the next version of the perfect support for Chinese search.
@luceos

clarkwinkelmann

samtto we are still looking for guidance as to how. There are no planned changes at this time.

There's a lot of talk here, but I can't see if anything was identified as an easy change. Does changing the character length for search in the frontend is enough or did it also require changes to the database side to make that work ?

It would be great if we could have the help of some Chinese developers on this. We'll need PRs or concrete code change suggestions.

Or at the very least a list of test cases.

Littlegolden

clarkwinkelmann There is an extension for support Chinese characters searching（based on and require Xunsearch）, hope it helps:
https://github.com/jjandxa/flarum-ext-chinese-search

clarkwinkelmann

Littlegolden nice one! Would be cool if the author published it under our Extensions tag 😃

I wonder how the performance compares with the MySQL-based search.

danielunited

Does this also improvoe search for Hebrew characters?
If not, is there any other way to do it?

Littlegolden

danielunited Yes, I think so

stimw

Littlegolden well...improving search will be in 2.0 (flarum/core1355)

It will be a long process. ☹️

Have anyone tested this extension: https://github.com/mkm0094/flarum-typesense-chinese-search

Hari

samtto Support Chinese search

i think this is improved Littlegolden

qin-yu

It looks like this open issue has been there for about 5 years but archived... flarum/issue-archive286

Does anyone know when Flarum will eventually get a search engine? I'm about to abandon it but still hope a useable search fucntion will be integrated into it before my community are too upset about my choice of using Flarum.

stimw I can't find where it says the new thing can be introduced in 2.0 😢

lichengkun135

Littlegolden Hi, I'm wondering can this method work with freeflarum?

luceos

qin-yu did you see this? It offers some solutions: https://discuss.flarum.org.cn/d/3994. I think the scout extension with meili would be a simple alternative now. Or check some of the answers above.

qin-yu It looks like this open issue has been there for about 5 years but archived... flarum/issue-archive#286

Most of the changes that issue talks about are implemented. We had an issue for UTF8 compatibility, but aside from the MySQL configuration required only a setting would be required in Flarum to allow configuration of the minimum number of search input characters, which is now hardcoded to 3.

come2mejk

qin-yu

About chinese search solution

1、Sonic

2、Meilisearch

3、Algolia

4、Elasticsearch

https://discuss.flarum.org.cn/d/3994

« Previous Page