Realtime - self hosted websocket for instant updates

luceos

Blomstra Realtime is now available for everyone to subscribe to. This websocket extension that powers the experience of realtime on Discuss has been created to offer a self-hosted alternative to the Flarum Pusher extension. It's extremely scalable, tested with thousands of concurrent users 💪

Starting development last year and writing everything from scratch, including the actual operation of a websocket server, we are overly excited to bring this extension to the ecosystem. Due to the complexity of this extension and the hours invested to make this scalable, bug free and performant while promising to do this moving forward, we cannot provide it for free, but I hope we made it interesting for everyone regardless.

There are two plans, read the OP for all the details, but in a nutshell:

A not-for-profit plan where you take care of installation and configuration at a very low, one-off lifetime cost.
A for-profit plan with support, including hands-on for installation, at a recurring subscription.

The plans are available through links in the OP only, because I want to prevent people from subscribing to the not-for-profit license without understanding what they signed up for 😉

If you have any questions, let me know.
If you are already using realtime, share the love 🙂

mekici

luceos Can you make a setup document for the LEMP server? I still can't get it to work 🙂

This is how I configured it.
Fixed the delayed notification issue in the notification bell with 1.5, right?

pkernstock

luceos If you are already using realtime, share the love 🙂

I've been using realtime for a forum about an open-source project since the early beta days, which should be about a year by now. I'm really much enjoying the real-time experience (pun intended) the extension adds to the community forum, and obviously also the support and efforts @luceos has been providing during the setup-phase. Since then, the extension does a great job without any issues.

Just one thing what I've been forgetting in early days: Restarting the realtime service on the server after an update of Flarum or the extension.

Darkle

mekici Can you test what happens if instead of location @ws { you use location /app {?

Kulga

I seem to be having some issues installing this.
What I'm doing:

docker container run --rm -it webdevops/php-nginx:8.1 bash
cd /app
composer create-project flarum/flarum .
composer config repositories.extiverse composer https://extiverse.com/composer/
composer config --global --auth bearer.extiverse.com AAAAAA....mytoken
composer require -W blomstra/realtime:"*"

I then get this error:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - blomstra/realtime 0.2 requires pusher/pusher-php-server ^4.1.5 -> found pusher/pusher-php-server[v4.1.5] but these were not loaded, likely because it conflicts with another require.
    - blomstra/realtime[0.2.1, ..., 0.2.9] require pusher/pusher-php-server ^7.0.1 -> found pusher/pusher-php-server[7.0.1, 7.0.2] but these were not loaded, likely because it conflicts with another require.
    - Root composer.json requires blomstra/realtime * -> satisfiable by blomstra/realtime[0.2, ..., 0.2.9].


Installation failed, reverting ./composer.json and ./composer.lock to their original content.

composer self-update
You are already using the latest available Composer version 2.4.2 (stable channel).

Justoverclock

Kulga uninstall pusher maybe

mekici

Kulga Uninstall pusher with the core extension. Then try again.
composer remove flarum/pusher

luceos

Released 0.2.10:

fix: activity is pushed into users page discussions
add nginx include file for simpler use
updated readme with configuration options and nginx include file

It should now be much easier to configure Realtime with Nginx. You don't need any Upgrade parameters outside of your server block, just add the include or copy/paste it into the existing server block 👍️

Kulga as the others mentioned, remove flarum/pusher and kyrne/websocket. Let us know whether that solves it for you.

Kulga

Yup, that resolved it

Lurker

pkernstock Restarting the realtime service on the server after an update of Flarum or the extension.

I know that with Redis queues, the command would be php flarum queue:restart. For Blomstra/Realtime, is there a specific command to restart the service likephp flarum realtime:restart?

Thanks! 👍

luceos

Release version 0.2.11, which upgrades the pusher dependencies to 7.2.0 for PHP and 7.4.0 for JS. Make sure to install 0.2.11 if you have updated with composer recently as it would automatically pull in these pusher dependencies.

luceos

Lurker sadly nothing like that exists, but it would be interesting to add.. You can restart the service after an update manually or give sudoers access to your user to restart the service without a password in ci/cd encironments.

Lurker

luceos Queue jobs would hit the timeout because the default timeout is 60s

Is this still the case for the most recent version of Realtime? I know that the default is set to 120s, but what would you say is the safe threshold for the interval? Thanks!

luceos

Lurker this does not relate to the interval, but the duration of the tasks to process notifications. And yes this issue was resolved. Are you having any issues?

Lurker

luceos I think so. As admin in my forum, I receive the web notifications but for my test user, it does not seem to appear. When I click the notifications icon, it is empty. I have to manually refresh the page for it to appear (greyed out). There is no web notification even though I tagged the test user and made it follow the discussion. The email notification seems to be working fine.

I have Redis and Realtime installed with both of them actively running and configured to start automatically using systemd.

I am only on Entry edition so I am not certain if support can be given on this matter. If it requires expert troubleshooting, then I would understand and just attempt to solve it myself. Cheers! 😃

luceos

Lurker if an admin sees realtime updates and a regular doesn't, then it's not an issue with the queue but perhaps of permissions... If you're able to narrow it down a little bit more I can look into it.

Lurker

luceos My apologies if it took awhile for me to respond. I did not know how to dispatch a queue until this morning.

Situation:
I have both admin user and test user log-in to the forum using different devices - laptop and mobile respectively. Test user creates a discussion, follows it and mentions admin user and vice-versa but neither of them get any web notifications. The bell (notification) icon does not glow orange and when you click on it, nothing is shown. But when you refresh the page, the notification appears (greyed out). Email notifications do get sent.

Background:
Realtime and Redis service are actively running in the background when I type sudo systemctl status. Both web and email notifications are active.

The Realtime service is using the default setting as I don't know how to configure the js-client, php-client and app while my Redis service is using this:

return [
    (new Blomstra\Redis\Extend\Redis([
        'host' => '127.0.0.1',
        'password' => null,
        'port' => 6379,
        'database' => 1,
    ]))
    ->useDatabaseWith('cache', 1)
    ->useDatabaseWith('queue', 2)
    ->useDatabaseWith('session', 3)
];

Assessment:
I went into the terminal and decided to check what's going on by typing sudo -u www-data php flarum queue:work -vvv. Test user makes a post and mentions admin user again. This is the output I see on the terminal:

[2022-11-09 16:19:12][mxOYTONEv33cmrIu6JhBHIe3CaCRyorj] Processing: Blomstra\Realtime\Push\Jobs\SendTriggerJob
[2022-11-09 16:20:27][mxOYTONEv33cmrIu6JhBHIe3CaCRyorj] Failed:     Blomstra\Realtime\Push\Jobs\SendTriggerJob
Killed

I don't know where the problem lies. I'm pretty sure I religiously followed all the instructions on the guide.

I hope the explanation is sufficient. Thanks!

luceos

@GreXXL discovered an issue with some of the config.php snippets in this discussion OP, the key sometimes was mistakenly realtime where it should have been websocket.

luceos

Lurker I don't know where the problem lies. I'm pretty sure I religiously followed all the instructions on the guide.

Failed means it couldn't send the notification to the pusher service. Have you tried updating your Flarum config.php with a websocket key?

For instance:

<?php return array (
  'debug' => false,
  'database' =>
  array (
    'driver' => 'mysql',
    'host' => 'localhost',
    'port' => 3306,
    'database' => 'flarumdb',
    'username' => 'flarum',
    'password' => '[snip]',
    'charset' => 'utf8mb4',
    'collation' => 'utf8mb4_unicode_ci',
    'prefix' => '',
    'strict' => false,
    'engine' => NULL,
    'prefix_indexes' => true,
  ),
  'url' => 'https://example.com/',
  'paths' =>
  array (
    'api' => 'api',
    'admin' => 'admin',
  ),
  'headers' =>
  array (
    'poweredByHeader' => true,
    'referrerPolicy' => 'same-origin',
  ),
  'websocket' => [
      'php-client-host' => 'localhost',
  ],
);

See if that helps.

Lurker

luceos Opened my config.php and added the recommended lines:

  ),
  'websocket' => [
      'php-client-host' => 'localhost',
  ],

Repeated the same setup mentioned before and got the following result:

[2022-11-10 04:00:18][4LgZZitrAvPNZgrHva5EGB0tknY4wR2U] Processing: Flarum\Subscriptions\Job\SendReplyNotification
[2022-11-10 04:00:18][4LgZZitrAvPNZgrHva5EGB0tknY4wR2U] Processed:  Flarum\Subscriptions\Job\SendReplyNotification
[2022-11-10 04:00:18][2bxprgsIFGAq2KqYeqThhOBhR9WbQSxu] Processing: Flarum\Notification\Job\SendNotificationsJob
[2022-11-10 04:00:18][2bxprgsIFGAq2KqYeqThhOBhR9WbQSxu] Processed:  Flarum\Notification\Job\SendNotificationsJob
[2022-11-10 04:00:18][SZG5760kGbTvqJrExDZOC9Sw0CN6VVzx] Processing: Blomstra\Realtime\Push\Jobs\SendNotificationsJob
[2022-11-10 04:02:48][SZG5760kGbTvqJrExDZOC9Sw0CN6VVzx] Failed:     Blomstra\Realtime\Push\Jobs\SendNotificationsJob
Killed

I have the websocket server running encrypted via nginx as per the guide. No errors encountered when I typed sudo nginx -t. It seems that the redis.service is able to process the task but it does not propagate to the realtime.service. 🤔

Edit:
Please ignore this message as this has been resolved after several days of analyses using my non-computer-savvy brain 😜

« Previous Page Next Page »