Redis sessions, cache & queues

Hiob · May 10, 2020

Hi @luceos, I just want to say thank you! You doing a great job, as usual.
Really simple to install, really effective!

For redis, I used the docker image bitnami / redis with docker-compose, it works like a charm.

mekici · May 10, 2020

root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13# php flarum queue:work


  There are no commands defined in the "queue" namespace.


root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13# php flarum queue:work -vvv

In Application.php line 597:

  [Symfony\Component\Console\Exception\NamespaceNotFoundException]
  There are no commands defined in the "queue" namespace.


Exception trace:
  at /var/www/beta13/vendor/symfony/console/Application.php:597
 Symfony\Component\Console\Application->findNamespace() at /var/www/beta13/vendor/symfony/console/Application.php:650
 Symfony\Component\Console\Application->find() at /var/www/beta13/vendor/symfony/console/Application.php:235
 Symfony\Component\Console\Application->doRun() at /var/www/beta13/vendor/symfony/console/Application.php:147
 Symfony\Component\Console\Application->run() at /var/www/beta13/vendor/flarum/core/src/Console/Server.php:44
 Flarum\Console\Server->listen() at /var/www/beta13/flarum:24

root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13#

I set extend.php as described. LEMP on the server
The redis server has been successfully installed but I get such an error. Where could the problem be?

luceos · May 10, 2020

mekici did you enable the extension in your admin area?

mekici · May 10, 2020

luceos yes i enabled but since i subscribed to the recache plugin this is not needed

https://discuss.flarum.org/d/20791-recache-speed-up-your-forum/126 can you help with this problem?

DursunCan · May 13, 2020

<?php

/*


 * This file is part of Flarum.
 *
 * (c) Toby Zerner <toby.zerner@gmail.com>
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

use Flarum\Extend;

return [
    // Register extenders here to customize your forum!
];

return [
    new Bokt\Redis\Extend\EnableRedisCache([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]),
];

return [
    new Bokt\Redis\Extend\EnableRedisQueue([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]),
];

php flarum queue:work

                                                           
  There are no commands defined in the "queue" namespace.

plugin is active. why might it be like this?

datitisev · May 13, 2020

DursunCan One cannot return multiple values using return. You are returning the initial empty array with return [ ].

After the use line, your extend.php should look like this. Note that I replaced the repeated config with a variable, this way you only have to change the values once.

return [
    new Bokt\Redis\Extend\EnableRedisCache($redis = [
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]),
    new Bokt\Redis\Extend\EnableRedisQueue($redis),
];

DursunCan · May 13, 2020

datitisev I wrote the command and it looks like the picture. Waiting. It's been over 5 minutes. Is this normal?

datitisev · May 13, 2020

DursunCan Yes. The queue:work command listens to the Redis DB for queue jobs and runs them when any are added. It will not end until you tell it to.

You probably want to set up a service that runs php flarum queue:work continuously like supervisord or systemctl.

DursunCan · May 22, 2020

luceos Bokt/flarum-redis2

Gatsu · Jun 3, 2020

luceos Thank you for this.
Do you suggest to activate both? (Queque and cache)

luceos · Jun 3, 2020

Gatsu depends on the size of your community and how active it is. Using the cache really can't hurt. Using the queue requires you to understand that it has to be restarted when updating flarum or when adding extensions.

Gatsu · Jun 3, 2020

luceos Thank you for the explanation. As I understand it, there should be no problem activating the Redis cache with Opcache, right?

luceos · Jun 3, 2020

Gatsu not at all, they're two different things. Opcache caches your php files (code), redis replaces the file cache for views, session etc.

rob006 · Jun 3, 2020

luceos redis replaces the file cache for views

Views are compiled to PHP files. Do you store them in redis too?

luceos · Jun 3, 2020

rob006 it replaces the complete Cache store in Flarum. I'd have to trace it back, but it should be the case, yes.

rob006 · Jun 3, 2020

luceos How this could be more efficient than saving and reading these files from disk? They're read-heavy, so they should already be in disk cache in RAM and reading them should be faster than querying redis. Not mention opcache which works out of the box for files saved on disk, but I'm not sure how you managed to make it work if you store these files in redis.

luceos · Jun 4, 2020

rob006 diving deeper into this, it seems that the FileViewFinder in fact uses the Filesystem directly ️

DursunCan · Jun 13, 2020

luceos Bokt/flarum-redis4

DursunCan · Jul 14, 2020

Redis version 6.0.5

When the site opened, these articles appeared:

Warning: stream_socket_enable_crypto(): SSL operation failed with code 1. OpenSSL Error messages: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed in /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php on line 246

Fatal error: Uncaught Laminas\HttpHandlerRunner\Exception\EmitterException: Output has been emitted previously; cannot emit response in /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/Exception/EmitterException.php:24 Stack trace: #0 /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/Emitter/SapiEmitterTrait.php(40): Laminas\HttpHandlerRunner\Exception\EmitterException::forOutputSent() #1 /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/Emitter/SapiEmitter.php(27): Laminas\HttpHandlerRunner\Emitter\SapiEmitter->assertNoPreviousOutput() #2 /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/RequestHandlerRunner.php(98): Laminas\HttpHandlerRunner\Emitter\SapiEmitter->emit() #3 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Server.php(42): Laminas\HttpHandlerRunner\RequestHandlerRunner->run() #4 /var/www/sourceturk/flarum/public/index.php(26): Flarum\Http\Server->listen() #5 {main} thrown in /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/Exception/EmitterException.php on line 24

extend.php

new Bokt\Redis\Extend\EnableRedisCache($redis = [
        'host' => '127.0.0.1',
        'scheme' => 'tls',
        'password' => 'pass',
        'port' => 6379,
        'database' => 1,
    ]),

redis.conf

tls-cert-file /etc/redis/server.crt
tls-key-file /etc/redis/server.key
tls-ca-cert-file /etc/redis/rootCA.crt
tls-dh-params-file /etc/redis/dhparam.pem
port 0
tls-port 6379
protected-mode yes
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised systemd
bind 127.0.0.1 ::1
pidfile /var/run/redis/redis.pid
loglevel notice
always-show-logo yes
dir /var/lib/redis
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename dump.rdb
requirepass pass
rename-command FLUSHDB ""
rename-command FLUSHALL ""
rename-command DEBUG ""
rename-command SHUTDOWN SHUTDOWN_MENOT
rename-command CONFIG ASC12_CONFIG

storage/logs

[2020-07-14 15:20:10] production.ERROR: Predis\Connection\ConnectionException: Error while switching to encrypted communication [tls://127.0.0.1:6379] in /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/AbstractConnection.php:155
Stack trace:
#0 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php(247): Predis\Connection\AbstractConnection->onConnectionError()
#1 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php(107): Predis\Connection\StreamConnection->tlsStreamInitializer()
#2 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/AbstractConnection.php(81): Predis\Connection\StreamConnection->createResource()
#3 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php(258): Predis\Connection\AbstractConnection->connect()
#4 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/AbstractConnection.php(180): Predis\Connection\StreamConnection->connect()
#5 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php(288): Predis\Connection\AbstractConnection->getResource()
#6 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/StreamConnection.php(394): Predis\Connection\StreamConnection->write()
#7 /var/www/sourceturk/flarum/vendor/predis/predis/src/Connection/AbstractConnection.php(110): Predis\Connection\StreamConnection->writeRequest()
#8 /var/www/sourceturk/flarum/vendor/predis/predis/src/Client.php(331): Predis\Connection\AbstractConnection->executeCommand()
#9 /var/www/sourceturk/flarum/vendor/predis/predis/src/Client.php(314): Predis\Client->executeCommand()
#10 /var/www/sourceturk/flarum/vendor/illuminate/redis/Connections/Connection.php(114): Predis\Client->__call()
#11 /var/www/sourceturk/flarum/vendor/illuminate/redis/Connections/Connection.php(214): Illuminate\Redis\Connections\Connection->command()
#12 /var/www/sourceturk/flarum/vendor/illuminate/cache/RedisStore.php(54): Illuminate\Redis\Connections\Connection->__call()
#13 /var/www/sourceturk/flarum/vendor/illuminate/cache/Repository.php(97): Illuminate\Cache\RedisStore->get()
#14 /var/www/sourceturk/flarum/vendor/illuminate/cache/Repository.php(366): Illuminate\Cache\Repository->get()
#15 /var/www/sourceturk/flarum/vendor/fof/terms/src/Repositories/PolicyRepository.php(38): Illuminate\Cache\Repository->rememberForever()
#16 /var/www/sourceturk/flarum/vendor/fof/terms/src/Extenders/ForumPoliciesRelationship.php(33): FoF\Terms\Repositories\PolicyRepository->all()
#17 /var/www/sourceturk/flarum/vendor/illuminate/events/Dispatcher.php(360): FoF\Terms\Extenders\ForumPoliciesRelationship->loadRelationship()
#18 /var/www/sourceturk/flarum/vendor/illuminate/events/Dispatcher.php(209): Illuminate\Events\Dispatcher->Illuminate\Events\{closure}()
#19 /var/www/sourceturk/flarum/vendor/flarum/core/src/Api/Controller/AbstractSerializeController.php(99): Illuminate\Events\Dispatcher->dispatch()
#20 /var/www/sourceturk/flarum/vendor/flarum/core/src/Api/Client.php(72): Flarum\Api\Controller\AbstractSerializeController->handle()
#21 /var/www/sourceturk/flarum/vendor/flarum/core/src/Frontend/Frontend.php(72): Flarum\Api\Client->send()
#22 /var/www/sourceturk/flarum/vendor/flarum/core/src/Frontend/Frontend.php(51): Flarum\Frontend\Frontend->getForumDocument()
#23 /var/www/sourceturk/flarum/vendor/flarum/core/src/Frontend/Controller.php(32): Flarum\Frontend\Frontend->document()
#24 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/RouteHandlerFactory.php(38): Flarum\Frontend\Controller->handle()
#25 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/DispatchRoute.php(65): Flarum\Http\RouteHandlerFactory->Flarum\Http\{closure}()
#26 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\DispatchRoute->process()
#27 /var/www/sourceturk/flarum/vendor/kilowhat/flarum-ext-audit-free/src/Middlewares/SetLoggerActor.php(28): Laminas\Stratigility\Next->handle()
#28 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Kilowhat\Audit\Middlewares\SetLoggerActor->process()
#29 /var/www/sourceturk/flarum/vendor/fof/terms/src/Middlewares/RegisterMiddleware.php(54): Laminas\Stratigility\Next->handle()
#30 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): FoF\Terms\Middlewares\RegisterMiddleware->process()
#31 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/ShareErrorsFromSession.php(57): Laminas\Stratigility\Next->handle()
#32 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\ShareErrorsFromSession->process()
#33 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/SetLocale.php(50): Laminas\Stratigility\Next->handle()
#34 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\SetLocale->process()
#35 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/CheckCsrfToken.php(37): Laminas\Stratigility\Next->handle()
#36 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\CheckCsrfToken->process()
#37 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/AuthenticateWithSession.php(32): Laminas\Stratigility\Next->handle()
#38 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\AuthenticateWithSession->process()
#39 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/RememberFromCookie.php(51): Laminas\Stratigility\Next->handle()
#40 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\RememberFromCookie->process()
#41 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/StartSession.php(61): Laminas\Stratigility\Next->handle()
#42 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\StartSession->process()
#43 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/CollectGarbage.php(46): Laminas\Stratigility\Next->handle()
#44 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\CollectGarbage->process()
#45 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/ParseJsonBody.php(28): Laminas\Stratigility\Next->handle()
#46 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\ParseJsonBody->process()
#47 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Middleware/HandleErrors.php(57): Laminas\Stratigility\Next->handle()
#48 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Flarum\Http\Middleware\HandleErrors->process()
#49 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(84): Laminas\Stratigility\Next->handle()
#50 /var/www/sourceturk/flarum/vendor/middlewares/request-handler/src/RequestHandler.php(84): Laminas\Stratigility\MiddlewarePipe->process()
#51 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Middlewares\RequestHandler->process()
#52 /var/www/sourceturk/flarum/vendor/middlewares/base-path-router/src/BasePathRouter.php(97): Laminas\Stratigility\Next->handle()
#53 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Middlewares\BasePathRouter->process()
#54 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Middleware/OriginalMessages.php(42): Laminas\Stratigility\Next->handle()
#55 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Laminas\Stratigility\Middleware\OriginalMessages->process()
#56 /var/www/sourceturk/flarum/vendor/middlewares/base-path/src/BasePath.php(53): Laminas\Stratigility\Next->handle()
#57 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/Next.php(61): Middlewares\BasePath->process()
#58 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(84): Laminas\Stratigility\Next->handle()
#59 /var/www/sourceturk/flarum/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(73): Laminas\Stratigility\MiddlewarePipe->process()
#60 /var/www/sourceturk/flarum/vendor/laminas/laminas-httphandlerrunner/src/RequestHandlerRunner.php(96): Laminas\Stratigility\MiddlewarePipe->handle()
#61 /var/www/sourceturk/flarum/vendor/flarum/core/src/Http/Server.php(42): Laminas\HttpHandlerRunner\RequestHandlerRunner->run()
#62 /var/www/sourceturk/flarum/public/index.php(26): Flarum\Http\Server->listen()

sudo systemctl status redis

● redis.service - Advanced key-value store
   Loaded: loaded (/etc/systemd/system/redis.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2020-07-14 15:19:04 UTC; 35min ago
     Docs: http://redis.io/documentation,
           man:redis-server(1)
 Main PID: 3116 (redis-server)
    Tasks: 4 (limit: 4659)
   CGroup: /system.slice/redis.service
           └─3116 /usr/local/bin/redis-server 127.0.0.1:6379

Jul 14 15:19:03 roots systemd[1]: Starting Advanced key-value store...
Jul 14 15:19:03 roots redis-server[3113]: 3113:C 14 Jul 2020 15:19:03.999 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
Jul 14 15:19:04 roots redis-server[3113]: 3113:C 14 Jul 2020 15:19:04.000 # Redis version=6.0.5, bits=64, commit=00000000, modified=0, pid=3113, just started
Jul 14 15:19:04 roots redis-server[3113]: 3113:C 14 Jul 2020 15:19:04.000 # Configuration loaded
Jul 14 15:19:04 roots systemd[1]: Started Advanced key-value store.

luceos · Aug 6, 2020

DursunCan never responded to this, but this is a server/configuration issue. There's nothing I can do. Perhaps fall back to the non-tls connection.