Database Queue - the simplest queue, even for shared hosting

luceos

poVoq I don't know what your PHP script does and honestly if you hit a web endpoint from a Cron on another machine you will possibly just hit timeouts continuously. I think it's too unreliable. The hoster might interpret the activity as unwanted too for instance.

Header_Footer

Sir Thank You so much for this extension, It boosted my posting speed before when i click post reply it takes at least 5 seconds and now in milliseconds post is posted.

Sir as I can see it's in beta stage so is it safe to use it in production and also running a cron every minute will add load on server or not, I am using a 4 core cpu? 16 gb ram

luceos

Header_Footer the cronjob that checks for background tasks to process does not add any noticeable load to the server by itself. When processing tasks it actually does so efficiently while taking that load away from the interaction that causes those tasks to be fired. So netto/bottomline the use of a queue is more efficient than without it.

Hari

luceos unable to enable DB queues extension, do you think it is because I'm using MySQL 5.6?

Edit : for now I want to try againRedis queue using corn job like mentioned here rafaucau

I will keep you posted. thanks for responding. 🌼

luceos

Hari check the logs for the actual error or enable debug. I cannot tell what the problem is with just the fact that an error ocurred.

Braden

Installed

composer require blomstra/database-queue:^0.1-beta.1

Activated

In logs
flarum.ERROR: Symfony\Component\Console\Exception\NamespaceNotFoundException: There are no commands defined in the "queue" namespace. in /public_html/vendor/symfony/console/Application.php:612 Stack trace: #0 /public_html/vendor/symfony/console/Application.php(663): Symfony\Component\Console\Application->findNamespace('queue') #1 /public_html/vendor/symfony/console/Application.php(255): Symfony\Component\Console\Application->find('queue:work') #2 /public_html/vendor/symfony/console/Application.php(167): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput)) #3 /public_html/vendor/flarum/core/src/Console/Server.php(42): Symfony\Component\Console\Application->run() #4 /public_html/flarum(24): Flarum\Console\Server->listen() #5 {main}
It could be a minor issue?

luceos

Braden what is the output of php flarum info?

Braden

Flarum core 1.0.4
PHP version: 8.0.8
Loaded extensions: Core, date, libxml, openssl, pcre, zlib, filter, hash, json, pcntl, readline, Reflection, SPL, session, standard, bcmath, bz2, calendar, ctype, curl, dba, dom, enchant, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, imap, intl, ldap, exif, mysqlnd, odbc, PDO, pgsql, Phar, posix, pspell, shmop, SimpleXML, snmp, soap, sockets, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, tokenizer, xml, xmlwriter, xsl, zip, mysqli, pdo_mysql, PDO_ODBC, pdo_pgsql, pdo_sqlite, xmlreader, redis, Zend OPcache
+------------------------------+------------+--------+
| Flarum Extensions            |            |        |
+------------------------------+------------+--------+
| ID                           | Version    | Commit |
+------------------------------+------------+--------+
| flarum-flags                 | v1.0.0     |        |
| flarum-suspend               | v1.0.0     |        |
| flarum-likes                 | v1.0.0     |        |
| flarum-approval              | v1.0.0     |        |
| flarum-tags                  | v1.0.3     |        |
| blomstra-database-queue      | 0.1-beta.1 |        |
| fof-drafts                   | 1.0.1      |        |
| noriods-auto-more            | v1.0.0     |        |
| clarkwinkelmann-emojionearea | 1.0.0      |        |
| fof-best-answer              | 1.0.1      |        |
| fof-nightmode                | 1.0.0      |        |
| fof-reactions                | 1.0.0      |        |
| fof-user-bio                 | 1.0.0      |        |
| jslirola-login2seeplus       | v0.2       |        |
| dem13n-topic-starter-label   | 0.1.6      |        |
| flarum-sticky                | v1.0.0     |        |
| fof-bbcode-details           | 1.0.0      |        |
| flarum-markdown              | v1.0.1     |        |
| flarum-pusher                | v1.0.0     |        |
| flarum-bbcode                | v1.0.0     |        |
| justoverclock-welcomebox     | 1.2.9      |        |
| fof-linguist                 | 1.0.0      |        |
| fof-oauth                    | 1.0.1      |        |
| fof-analytics                | 1.0.0      |        |
| flarum-lang-english          | v1.0.0     |        |
| zerosonesfun-bbcode-audio    | 2.1        |        |
| fof-formatting               | 1.0.0      |        |
| antoinefr-money              | v1.1.0     |        |
| flarum-mentions              | v1.0.0     |        |
| v17development-seo           | v1.8.0     |        |
| flarum-subscriptions         | v1.0.0     |        |
| v17development-user-badges   | v0.2.1     |        |
| clarkwinkelmann-mailing      | 1.0.0      |        |
| flarum-lock                  | v1.0.0     |        |
| fof-username-request         | 1.0.0      |        |
| fof-share-social             | 1.0.0      |        |
| fof-polls                    | 1.0.3      |        |
| fof-links                    | 1.0.0      |        |
| flarum-emoji                 | v1.0.0     |        |
| fof-ban-ips                  | 1.0.0      |        |
| fof-disposable-emails        | 1.0.0      |        |
| fof-ignore-users             | 1.0.0      |        |
+------------------------------+------------+--------+

010101

A tip for webmasters who may forget they set this up. 😊 If your community stops getting notifications it’s likely this. And more specifically, in my case what happened is I started to have to use php80 for any php command line directives due to my hosting set up to force php 8+. I was getting a composer dependencies don’t match your php version error every second in my php error log. My host was very nice and pointed out to me that I needed to edit my cron job rule related to this extension.

php80 flarum queue:work --stop-when-empty

Again, note the php80 part.

Now I just have to remember to change this back once my web host switches their shared servers to php 8+ by default. And possibly again way in the future for php 9+. 🙃

CyberGene

What are the downsides of using this queue instead of the default sync driver for a small community (150 users, 10-15 posts a day)? In other words, what types of events are dispatched to the queue? Because the cron would be executed once every minute, this means some events that previously would be dispatched immediately, would now be delayed for up to 1 minute, right? Which ones? If it's mails, that's OK but I'm wondering if this will affect regular notifications? Will Pusher be affected or it's a separate functionality? As a side question, will the forum stats widget benefit from the queue? I notice that it loads its stats on every page load and takes some time, so if there's some pre-calculated state working on the queue and the stats widget only fetching it instead of querying the DB each time, it would be great.

luceos

CyberGene Because the cron would be executed once every minute, this means some events that previously would be dispatched immediately, would now be delayed for up to 1 minute, right?

Correct.

CyberGene Will Pusher be affected or it's a separate functionality?

Yes also pusher will be affected.

CyberGene As a side question, will the forum stats widget benefit from the queue?

If it uses the queue yes. It probably could use the cache, a queue isn't even that necessary I think.

CyberGene What are the downsides of using this queue instead of the default sync driver for a small community (150 users, 10-15 posts a day)?

In general there are only benefits. But as you pointed out there are some considerations. You could use the redis queue which is instant by default. Or you could try to run the work command with a non overlapping daemon runner if using the database queue to make it near instant too.

FinKiin

~~I can't receive any notice in my flarum~~
Never mind,solved

Oshvam

I have the ability to use Redis on my shared server. The problem is that activating it together with the Redis Cache extension from Flarum generates me a lot of problems with queues and usually pauses notifications.

Would it be more advisable to use this extension instead?

Subarist

What do I need to check?

`[root@instance20220311042805 flarum]# php flarum queue:work
PHP Fatal error:  Uncaught Error: Call to undefined function Illuminate\Queue\pcntl_alarm() in /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php:228
Stack trace:
#0 /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php(163): Illuminate\Queue\Worker->registerTimeoutHandler()
#1 /www/wwwroot/flarum/vendor/illuminate/queue/Console/WorkCommand.php(116): Illuminate\Queue\Worker->daemon()
#2 /www/wwwroot/flarum/vendor/illuminate/queue/Console/WorkCommand.php(100): Illuminate\Queue\Console\WorkCommand->runWorker()
#3 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#4 /www/wwwroot/flarum/vendor/illuminate/container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#5 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#6 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#7 /www/wwwroot/flarum/vendor/illuminate/container/Container.php(653): Illuminate\Container\BoundMethod::call()
#8 /www/wwwroot/flarum/vendor/illuminate/console/Command.php(136): Illuminate\Container\Container->call()
#9 /www/wwwroot/flarum/vendor/symfony/console/Command/Command.php(298): Illuminate\Console\Command->execute()
#10 /www/wwwroot/flarum/vendor/illuminate/console/Command.php(120): Symfony\Component\Console\Command\Command->run()
#11 /www/wwwroot/flarum/vendor/symfony/console/Application.php(1058): Illuminate\Console\Command->run()
#12 /www/wwwroot/flarum/vendor/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand()
#13 /www/wwwroot/flarum/vendor/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun()
#14 /www/wwwroot/flarum/vendor/flarum/core/src/Console/Server.php(42): Symfony\Component\Console\Application->run()
#15 /www/wwwroot/flarum/flarum(24): Flarum\Console\Server->listen()
#16 {main}
  thrown in /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php on line 228

Fatal error: Uncaught Error: Call to undefined function Illuminate\Queue\pcntl_alarm() in /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php:228
Stack trace:
#0 /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php(163): Illuminate\Queue\Worker->registerTimeoutHandler()
#1 /www/wwwroot/flarum/vendor/illuminate/queue/Console/WorkCommand.php(116): Illuminate\Queue\Worker->daemon()
#2 /www/wwwroot/flarum/vendor/illuminate/queue/Console/WorkCommand.php(100): Illuminate\Queue\Console\WorkCommand->runWorker()
#3 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(36): Illuminate\Queue\Console\WorkCommand->handle()
#4 /www/wwwroot/flarum/vendor/illuminate/container/Util.php(40): Illuminate\Container\BoundMethod::Illuminate\Container\{closure}()
#5 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(93): Illuminate\Container\Util::unwrapIfClosure()
#6 /www/wwwroot/flarum/vendor/illuminate/container/BoundMethod.php(35): Illuminate\Container\BoundMethod::callBoundMethod()
#7 /www/wwwroot/flarum/vendor/illuminate/container/Container.php(653): Illuminate\Container\BoundMethod::call()
#8 /www/wwwroot/flarum/vendor/illuminate/console/Command.php(136): Illuminate\Container\Container->call()
#9 /www/wwwroot/flarum/vendor/symfony/console/Command/Command.php(298): Illuminate\Console\Command->execute()
#10 /www/wwwroot/flarum/vendor/illuminate/console/Command.php(120): Symfony\Component\Console\Command\Command->run()
#11 /www/wwwroot/flarum/vendor/symfony/console/Application.php(1058): Illuminate\Console\Command->run()
#12 /www/wwwroot/flarum/vendor/symfony/console/Application.php(301): Symfony\Component\Console\Application->doRunCommand()
#13 /www/wwwroot/flarum/vendor/symfony/console/Application.php(171): Symfony\Component\Console\Application->doRun()
#14 /www/wwwroot/flarum/vendor/flarum/core/src/Console/Server.php(42): Symfony\Component\Console\Application->run()
#15 /www/wwwroot/flarum/flarum(24): Flarum\Console\Server->listen()
#16 {main}
  thrown in /www/wwwroot/flarum/vendor/illuminate/queue/Worker.php on line 228
[root@instance20220311042805 flarum]# `


Flarum core: 1.7.2
PHP version: 8.2.4
MySQL version: 8.0.33
Loaded extensions: Core, date, libxml, openssl, pcre, sqlite3, zlib, bcmath, ctype, curl, dom, filter, ftp, gd, gettext, hash, iconv, intl, json, mbstring, SPL, session, pcntl, standard, mysqlnd, PDO, pdo_mysql, pdo_sqlite, Phar, posix, random, Reflection, mysqli, shmop, SimpleXML, soap, sockets, sodium, sysvsem, tokenizer, xml, xmlreader, xmlwriter, zip, fileinfo, memcached, redis, exif, gmp, Zend OPcache
+-------------------------------------+---------------+--------+
| Flarum Extensions                   |               |        |
+-------------------------------------+---------------+--------+
| ID                                  | Version       | Commit |
+-------------------------------------+---------------+--------+
| flarum-markdown                     | v1.7.0        |        |
| flarum-flags                        | v1.7.0        |        |
| flarum-suspend                      | v1.7.0        |        |
| askvortsov-rich-text                | v2.1.7        |        |
| flarum-tags                         | v1.7.1        |        |
| flarum-lock                         | v1.7.0        |        |
| flarum-sticky                       | v1.7.0        |        |
| fof-formatting                      | 1.0.3         |        |
| afrux-forum-widgets-core            | v0.1.7        |        |
| flarum-nicknames                    | v1.7.0        |        |
| fof-spamblock                       | 1.1.0         |        |
| fof-pages                           | 1.0.5         |        |
| flarum-approval                     | v1.7.0        |        |
| sycho-advanced-extension-categories | v0.1.3        |        |
| zequeen-acgembed-remastered         | v1.1.2        |        |
| v17development-seo                  | v1.8.0        |        |
| v17development-blog                 | v0.6.5        |        |
| the-turk-stickiest                  | 3.0.1         |        |
| sycho-profile-cover                 | v1.3.3        |        |
| sycho-lite-youtube                  | v0.1.1        |        |
| rob006-last-post-avatar             | 1.2.4         |        |
| pipecraft-id-slug                   | v1.1.0        |        |
| nearata-tags-color-generator        | v2.2.0        |        |
| malago-fancybox                     | 0.3.0         |        |
| justoverclock-hashtag               | 1.0.1         |        |
| justoverclock-custom-html-widget    | 0.1.4         |        |
| justoverclock-country-flags         | 0.2.1         |        |
| justoverclock-contactme             | 0.2.3         |        |
| ianm-html-head                      | 1.1.0         |        |
| fof-username-request                | 1.1.2         |        |
| fof-user-directory                  | 1.2.3         |        |
| fof-user-bio                        | 1.1.1         |        |
| fof-upload                          | 1.2.3         |        |
| fof-stopforumspam                   | 1.2.3         |        |
| fof-socialprofile                   | 1.1.5         |        |
| fof-sitemap                         | 2.1.0         |        |
| fof-share-social                    | 1.1.2         |        |
| fof-profile-image-crop              | 1.1.1         |        |
| fof-pretty-mail                     | 1.1.1         |        |
| fof-oauth                           | 1.3.0         |        |
| fof-nightmode                       | 1.5.1         |        |
| fof-merge-discussions               | 1.3.1         |        |
| fof-links                           | 1.1.4         |        |
| fof-drafts                          | 1.2.2         |        |
| fof-default-user-preferences        | 1.2.0         |        |
| fof-best-answer                     | 1.2.5         |        |
| fof-analytics                       | 1.1.0         |        |
| flarumtr-mobile-search              | v1.2          |        |
| flarum-subscriptions                | v1.7.0        |        |
| flarum-statistics                   | v1.7.0        |        |
| flarum-pusher                       | v1.7.0        |        |
| flarum-package-manager              | v1.0.0-beta.2 |        |
| flarum-mentions                     | v1.7.0        |        |
| flarum-likes                        | v1.7.0        |        |
| flarum-lang-english                 | v1.7.0        |        |
| flarum-lang-chinese-traditional     | v1.9.3        |        |
| flarum-lang-chinese-simplified      | v1.3.0        |        |
| flarum-emoji                        | v1.7.0        |        |
| flarum-bbcode                       | v1.7.0        |        |
| extiverse-mercury                   | 0.2.0         |        |
| davwheat-ads                        | 1.2.0         |        |
| datlechin-silent-edit               | v0.1.0        |        |
| datlechin-scroll-buttons            | v1.1.0        |        |
| datlechin-link-preview              | v1.3.2        |        |
| datlechin-chatgpt                   | v0.2.0        |        |
| clarkwinkelmann-scout               | 0.3.3         |        |
| clarkwinkelmann-first-post-approval | 1.0.1         |        |
| clarkwinkelmann-emojionearea        | 1.0.0         |        |
| clarkwinkelmann-author-change       | 1.0.3         |        |
| blomstra-turnstile                  | 0.1.2         |        |
| blomstra-spam-prevention            | 1.3.1         |        |
| blomstra-fontawesome                | 0.1.5         |        |
| blomstra-database-queue             | 0.1-beta.2    |        |
| askvortsov-pwa                      | v3.1.2        |        |
| askvortsov-markdown-tables          | v1.2.1        |        |
| afrux-online-users-widget           | v0.1.6        |        |
| afrux-forum-stats-widget            | v0.1.1        |        |
| afrux-asirem                        | v0.1.7        |        |
| acpl-mobile-tab                     | 1.1.1         |        |
+-------------------------------------+---------------+--------+
Base URL: https://www.subarist.net
Installation path: /www/wwwroot/flarum
Queue driver: database
Session driver: file
Scheduler status: 啟用
Mail driver: smtp
Debug mode: off

luceos

Subarist the php module pcntl needs to be installed and enabled.

Subarist

luceos
Tks,I have confirmed that the pcntl module is enabled .
Anyway, it seems to have returned to normal xd....
I don't know why, there are no longer any error reports now .

ctml

Curious if a queue takes longer than 1 minute to run, and the next cron is run would there be any conflict or does the next queue job wait for the current to complete or skip if an existing queue is being processed? Wondering if it would be better to setup using a systemd timer to avoid the possibility of running 2 jobs in parallel or if its mitigated by the extension? Thanks!

IanM

1.0.0

No longer require a cron to queue:work, simply uses the scheduler ensure the scheduler cron is running (better support for shared hosting where cli access is limited/difficult)
Display a simple queue status on the admin dashboard

blomstra/flarum-ext-database-queue3

Install/Updating

composer require blomstra/database-queue:"*"
php flarum cache:clear

ctml

IanM glad to see the queue extension moved to an official extension, thank you 🙂

When I used to invoke the queue using flarum queue:work, there were some arguments it supported I think such as --sleep, --tries, --memory, is it possible/necessary to configure those still if upgrading and allowing the scheduler to invoke the queue? Reason being, I need to ensure it has enough memory and by default I believe it's granted 128 MB, but I need to bump that up.

MikeLundahl

IanM Hi, I'm a little confused. You mentioned that no cron is required anymore. But the documents still say:

Set up

Go into your hosting control panel and set up the following task to run every minute:

php flarum schedule:run
Or in cron language:

* * * * cd /path/to/flarum && php flarum schedule:run

Is it the documents/readme that haven't been updated? Just want to make sure things are setup correctly. (I just updated flarum and its extensions).

« Previous Page Next Page »