Redis sessions, cache & queues

This library allows using Redis as cache, session and for the queue. You can only enable these services by using a local extender (the extend.php in the root of your Flarum installation). See the "Set up" section below.

This is an advanced utility for webmasters! It is not an extension and won't be visible in the admin area. Configuration can be done using a local extender or the glowingblue redis setup extension.

Installation

Install manually with composer:

composer require blomstra/flarum-redis:*

Set up

In your extend.php:

return [
    new Blomstra\Redis\Extend\Redis([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ])
];

This enables sessions, cache and queue to run on redis.

See "Use different database for each service" below to split up the database for cache vs sessions, queue, because a cache clear action will clear sessions and queue jobs as well if they share the same database.

Advanced configuration

  1. Disable specific services:
return [
    (new Blomstra\Redis\Extend\Redis([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]))->disable(['cache', 'queue'])
];
  1. Use different database for each service:
return [
    (new Blomstra\Redis\Extend\Redis([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]))
    ->useDatabaseWith('cache', 1)
    ->useDatabaseWith('queue', 2)
    ->useDatabaseWith('session', 3)
];
  1. Completely separate the config array:
return [
    (new Blomstra\Redis\Extend\Redis([
        'connections' => [
            'cache' => [
              'host' => 'cache.int.yoursite.com',
              'password' => 'foo-bar',
              'port' => 6379,
              'database' => 1,
            ],
            'queue' => [
              'host' => 'queue.int.yoursite.com',
              'password' => 'foo-bar',
              'port' => 6379,
              'database' => 1,
            ],
            'session' => [
              'host' => 'session.int.yoursite.com',
              'password' => 'foo-bar',
              'port' => 6379,
              'database' => 1,
            ],
        ],
    ]))
];

Queue

Make sure to start your queue workers, see the laravel documentation for specifics.

To test the worker can start use php flarum queue:work.

Updating

composer update blomstra/flarum-redis

Make sure to restart your queue workers every time you upgrade flarum, add or remove extensions. So whenever you ran composer commands you have to restart the queue.

FAQ

How do I configure a redis socket?

Use the following configuration:

[
       'scheme' => 'unix', 
       'path' => '<path to>redis.sock',
       'port' => 0,
       'database' => 1
]

Links


    • [deleted]

    @luceos can this be used with ReCache ?

      [deleted] if I'm not mistaken ReCache instantiates its own Redis store next to the one set up by Flarum. This extension forces Flarum to use Redis as a cache driver instead of the default file-based one. I see no reason why these should conflict, unless they use the same keys. You can circumvent that by using a different database setting than reCache does.

      You can selectively enable either Cache or Queue based on your needs by the way.

        • [deleted]

        luceos Thanks. ReCache also uses the same database by default, so I guess if you are using this (like I am) then you should at lease use a different database..

          [deleted] yeah, the example configuration actually originates from the Laravel documentation. The host, port and password need to match your instance. Redis automatically creates a new database once you use it.

            • [deleted]

            luceos Thanks. Off to try this out...

            • [deleted]

            Is there any simple way (apart from redis-cli monitor) to see if this is working ? Setup on my side, but seems very "quiet"

              [deleted] through ssh:

              $ php flarum queue:work -vvv

              Then dispatch something into the queue, if I'm not mistaken the fof/subscribed extension uses queues (ping @datitisev might know an easier way to test). You should now see an entry in your ssh terminal.

              • [deleted]

              Thanks.

              As a side note, this extension isn't only in use on the Bokt staging Flarum installation (not public), but also on discuss.flarum.org 👍️ . It powers follow-tags (among others) by pushing the delivery of email notifications into the queue. The benefit here for larger fora is that load times during post/discussion creation aren't affected.

                Is it possible to use this extension on shared hosting without shh

                  luceos so if I understand correctly, this is what would happen with queues (plus questions):

                  • 10 people are subscribed to a topic
                  • a user posts a new message in the topic
                  • instead of being sent immediately, the 10 emails are put into the Redis queue
                  • [what happens if the enqueue operation fails? is the message still posted without email notifications?]
                  • the long-running background Laravel process gets the items from the queue and processes them
                  • [what happens if the Laravel process is for some reason offline? are the queue items persisted for when the worker comes online, or does the system use pubsub so items are lost?]
                  • the queue worker sends the 10 email messages one by one
                  • [what happens if sending an email fails? is it retried or discarded? is there a limit of retries?]

                  Cheers!

                    Alkir Is it possible to use this extension on shared hosting without shh

                    No. You need access to a running redis instance. This extension is most definitely not for a regular shared hosting environment.

                    matteocontrini what happens if the enqueue operation fails?

                    If queueing of the Job fails, the request fails and the user gets direct feedback. If you use something like Sentry you will be notified as well. This erroring is exactly the same behavior with how it works by default in Flarum (called the sync driver). The message now is also still posted, but you will see an error (if I'm not mistaken).

                    matteocontrini what happens if the Laravel process is for some reason offline?

                    The job is queued nevertheless. If no workers are running no notifications are send. If you start a worker it will automatically pick up any items in the queue, starting from the oldest.

                    You can configure redis to persist its state to disk, so that the queue jobs are persisted to disk when redis or the server is restarted!

                    matteocontrini the queue worker sends the 10 email messages one by one

                    Queues allow easy scaling. You can run multiple workers at once, even remotely. As long as those remote servers can connect to the redis and database instance and run the same code/configuration. Queues are far more scalable than in-request processing (sync) of notifications.

                    matteocontrini what happens if sending an email fails?

                    You can configure this in the worker. php flarum queue:work --tries=4 will make the worker attempt jobs 4 times.

                    I can recommend using the fof/sentry package with this extension. This makes it easier to catch any issues with Jobs being processed in the queue.

                      6 months later

                      Hi @luceos, I just want to say thank you! You doing a great job, as usual. 😃
                      Really simple to install, really effective!

                      For redis, I used the docker image bitnami / redis with docker-compose, it works like a charm.

                      luceos

                      root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13# php flarum queue:work
                      
                      
                        There are no commands defined in the "queue" namespace.
                      
                      
                      root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13# php flarum queue:work -vvv
                      
                      In Application.php line 597:
                      
                        [Symfony\Component\Console\Exception\NamespaceNotFoundException]
                        There are no commands defined in the "queue" namespace.
                      
                      
                      Exception trace:
                        at /var/www/beta13/vendor/symfony/console/Application.php:597
                       Symfony\Component\Console\Application->findNamespace() at /var/www/beta13/vendor/symfony/console/Application.php:650
                       Symfony\Component\Console\Application->find() at /var/www/beta13/vendor/symfony/console/Application.php:235
                       Symfony\Component\Console\Application->doRun() at /var/www/beta13/vendor/symfony/console/Application.php:147
                       Symfony\Component\Console\Application->run() at /var/www/beta13/vendor/flarum/core/src/Console/Server.php:44
                       Flarum\Console\Server->listen() at /var/www/beta13/flarum:24
                      
                      root@lemp-s-1vcpu-2gb-fra1-01:/var/www/beta13#

                      I set extend.php as described. LEMP on the server
                      The redis server has been successfully installed but I get such an error. Where could the problem be?

                        <?php
                        
                        /*
                        
                        
                         * This file is part of Flarum.
                         *
                         * (c) Toby Zerner <toby.zerner@gmail.com>
                         *
                         * For the full copyright and license information, please view the LICENSE
                         * file that was distributed with this source code.
                         */
                        
                        use Flarum\Extend;
                        
                        return [
                            // Register extenders here to customize your forum!
                        ];
                        
                        return [
                            new Bokt\Redis\Extend\EnableRedisCache([
                                'host' => '127.0.0.1',
                                'password' => null,
                                'port' => 6379,
                                'database' => 1,
                            ]),
                        ];
                        
                        return [
                            new Bokt\Redis\Extend\EnableRedisQueue([
                                'host' => '127.0.0.1',
                                'password' => null,
                                'port' => 6379,
                                'database' => 1,
                            ]),
                        ];
                        php flarum queue:work
                        
                                                                                   
                          There are no commands defined in the "queue" namespace.  

                        plugin is active. why might it be like this?

                          DursunCan One cannot return multiple values using return. You are returning the initial empty array with return [ ].

                          After the use line, your extend.php should look like this. Note that I replaced the repeated config with a variable, this way you only have to change the values once.

                          return [
                              new Bokt\Redis\Extend\EnableRedisCache($redis = [
                                  'host' => '127.0.0.1',
                                  'password' => null,
                                  'port' => 6379,
                                  'database' => 1,
                              ]),
                              new Bokt\Redis\Extend\EnableRedisQueue($redis),
                          ];