Rich Embeds

CyberGene

Not sure if it's a bug with this extension or the Selective Media Embed (or some general incompatibility, for instance with the latest text formatter), however when I have YouTube disabled in the Selective Media Embed, hence going to this extension (Rich Embeds) for embedding, it just remains as a link. If I enable it, so that it get embedded by the regular Media Embed, then it works.

Also, we've been having some problems with embeds today, or maybe even for the last days.

As a side note, I updated my forum today and there were no new versions of extensions or Flarum, however some libs got updated.

P.S. I think it might be caused by a certain link in a post that messes up the Rich Embeds, for instance the entire Rich Embeds admin page hangs if I try to preview it:
https://www.stretta-music.de/gieseking-leimer-piano-technique-nr-184451.html

P.P.S. I don't know what's going on but no embeds work anymore. I am testing various links to popular websites and no preview can be generated for any of them.

clarkwinkelmann

CyberGene I'll look into this tomorrow.

Do you have any error message in the Flarum log file ?

Regarding YouTube, make sure you edit existing posts if they previously integrated using MediaEmbed. Despite looking like a regular link, my extension probably won't be able to convert a leftover/disabled MediaEmbed link until the post is edited and re-parsed. Make sure to also clear the Flarum cache and possibly your browser cache.

CyberGene

clarkwinkelmann Yes, I edit the posts (by adding an empty line, etc.) when testing the extension.

After a lot of experimentation yesterday, I believe the problem is due to the following link:
https://www.stretta-music.de/gieseking-leimer-piano-technique-nr-184451.html

There are no errors in the logs, however once that link is in a post, all other links in the entire discussion containing that post either disappear, or show empty rectangles, or work unpredictably. I guess it might be some uncaught parsing exception, or something like that, hopefully you can reproduce it.

clarkwinkelmann

Version 1.3.1 - January 4, 2024

Fixed embed failing to render and crashing Flarum if a website returns an invalid opengraph image URL.
Added handle additional URL formats in opengraph tags.

If you are updating from 1.3.0, you don't need to clear the forum cache as there have been no changes to the javascript or CSS.

You don't have to refresh the broken embeds to fix Flarum. But if the embed shows no images after the update, a manual refresh might make them appear.

CyberGene thanks for investigating. There were a few issues with the handling of URLs in my parser, and that website made the choice to use // protocol for its images, which I am not certain is allowed by OpenGraph but in any case my code should not have been crashing. Now those images are supported and if there is an invalid URL it will be properly skipped.

Let me know if that fixes all your issues or if I need to keep digging 😉

CyberGene

clarkwinkelmann thank you for the quick fix, it works 🙂

We have another problem, although I'm not sure it's with the Rich Embeds this time. See this link for example:

The Media Embed plugin is displaying just a white rectangle and can't show a proper Amazon preview. If I disable Amazon links with the Selective Media Embed and resort to the Rich Embeds, it also can't show any preview, just the raw link. I guess it's something Amazon changed on their end and broke OpenGraph?

P.S. Well, I have the same problem here with Discuss and there's white rectangle. The proper URL is contained HERE

CyberGene

@clarkwinkelmann I've been seeing a peculiar effect since the very beginning but never bothered posting about it since it’s very minor but here it is anyway. ~~All~~ Many links to forum’s own discussions appear with a picture that is my own avatar 😀 Why would that be? I’m the first user on the forum (as its creator) after the embedded admin user, if that’s a clue? Could it be that the extension just uses the very first image asset it finds on the file system and maybe that’s my avatar? Not a big deal anyway, because my avatar is a picture of grand piano keys which is what the forum is about but still, it’s a bit odd.

P.S. Actually that’s not always, just found some embeds are using pictures extracted from the discussion which is great.

clarkwinkelmann

CyberGene can you share with me the output of the command php flarum kilowhat:rich-embeds:process <url> -vvv where <url> is one of the URLs for which you see the issue?

For example php flarum kilowhat:rich-embeds:process https://discuss.flarum.org/d/30516-rich-embeds -vvv

Would then produce something like

Retrieved: 2024-02-04T21:25:12+00:00
HTTP Status: 200
OpenGraph: null
Icons: [
    {
        "href": "https:\/\/discuss.flarum.org\/assets\/favicon-hppi83fd.png"
    }
]
Fallback: {
    "title": "Rich Embeds - Flarum Community",
    "images": [
        {
            "alt": "Flarum Community",
            "src": "https:\/\/discuss.flarum.org\/assets\/logo-2q2rcevl.png"
        },
        {
            "alt": "Extiverse badge",
            "src": "https:\/\/extiverse.com\/extension\/kilowhat\/flarum-ext-rich-embeds\/badge"
        },
        {
            "alt": "Post with 2 different styles of embed",
            "src": "https:\/\/kilowhat.net\/medias\/extensions\/rich-embeds\/post.png"
        },
        {
            "src": "https:\/\/i.ibb.co\/4KcJG13\/DE630615-129-E-4910-BEBE-A28-D0-D710367.jpg"
        },
        {
            "src": "https:\/\/i.ibb.co\/L1TM09X\/F07580-F3-347-B-41-C3-AE9-C-7429251934-B0.jpg"
        },
        {
            "src": "https:\/\/i.ibb.co\/X2CPDYQ\/BDC9-B0-E7-F81-F-411-F-8-EDC-78278-C3-C0-EA9.jpg"
        },
        {
            "src": "https:\/\/i.imgur.com\/1qIUYsX.png"
        }
    ],
    "description": "Get support using and extending Flarum, the next-generation forum software that makes online discussion fun."
}
API Resource: null

You can sent it to my support email address if you would rather not share the URL here. But the payload won't contain anything confidential.

CyberGene

clarkwinkelmann the output of the command is here

P.S. Ohh, never mind, I got what it is. I’ve set that picture as “social media image” in the SEO extension. I’ll remove it there.

P.P.S. Why would sometimes the extension take a picture from the discussion instead? I like that and would prefer it over using the forum logo, if possible.

clarkwinkelmann

CyberGene glad you figured the first mystery out!

There are a number of fallback methods that impact which images are chosen:

First, if there is an opengraph image, it will take priority. This is what happens when the SEO extension is active
Second, if the forum advertises the X-Powered-By: Flarum header (Flarum does it by default), and the link points to the first post/page of the discussion, then the extension will intelligently extract images from the first post of the discussion
Finally, if the link is not recognized as Flarum or is not for a discussion, any image on the page will be picked, in the order they appear

If you experience a different behavior, let me know and I'll double-check if my explanation above is correct. I might have changed a few things with each update.

I could add a setting to blacklist specific image URLs, so that you could exclude a particular site logo if it doesn't look good. But if I find a way to make the parser better by default for everyone I will also try that. The problem is, it's difficult to guess what is a site logo or part of the content when doing what is already a fallback solution. The ideal solution would be for all sites to implement opengraph so they can choose which image they want to promote.

Clamours

Thank you very much for your great work, but sorry, is it possible for this extension to only enable rich image embedding functionality without enabling other features such as link previews? I only need the functionality to avoid image loading offsets and nothing else.
There are a large number of pictures in the discussion list and post content of my Flarum site (https://www.moresound.club/). In the iOS Safari browser, if I enter a discussion content page and then return to the homepage discussion list, the image will reload and cause serious offset. Jump offset, this problem is suspected to be caused by the lack of height label of the image. Can your extension solve this problem? Or where can I see this extension in action? Thank you again.

clarkwinkelmann

Clamours it's true that there is currently no separate settings, I should add more options and make the existing options clearer.

Looking at the code, the way things are currently implemented, the image parsing does not follow the general embed permission, so you should be able to achieve what you desire by forbidding members from using embeds. The images should still be parsed in their posts.

The image embed also disregards the "disable embeds on this post" feature. Even if someone who can post embeds then hides the link embeds, the image embed feature will continue to work.

Do you already have the extension? Let me know if that doesn't work and I'll try to add more options soon. If you don't have the extension yet, you can purchase it for testing and if it doesn't fit your needs I'll refund the purchase.

CyberGene

BTW, I used to hate Flarum for that scrolling issue on iPhone and it’s only now that I realize Rich Embeds solved it for me long ago 🥳 You rock @clarkwinkelmann

Darkle

I have migrated (server) and updated extensions, I have noticed that this extension doesn't work, the cached links work but there is no way to create any new ones.

In the admin preview when entering a test link it stays in "Retrieving...", I have no logs neither from Flarum/php, so it is complicated to debug what is happening, any idea?

clarkwinkelmann

Darkle please share your php flarum info

If you are able to, check the browser JavaScript console while using the preview feature and see if any error is generated there

Darkle

clarkwinkelmann I deleted this message, as I wasn't 100% sure:

I was testing with almost everything disabled, I seem to have found the problem, redis (when disabled it seems to work), I will share the details of the configuration.

But yeah, the problem is when using redis, I am using this extension (https://discuss.flarum.org/d/21873-redis-sessions-cache-queues), extend.php:

// Redis configuration
    (new Blomstra\Redis\Extend\Redis([
       'scheme' => 'unix', 
       'path' => '/tmp/redis.sock',
       'port' => 0,
       'database' => 1
    ])),

By deactivating redis and purging the extension, it works again.

clarkwinkelmann

Darkle it's probably not related to the cache, because only the Google Drive preview feature relies on the cache. Most likely it's related to the jobs running asynchronously. I would need to do tests on my own server to investigate.

If you have access to the queue log or the database, could you check if you see any indication of the job starting and/or completing? The job is called ProcessUrl. In the kilowhat_rich_embeds table, you could look for the URL that was entered in a preview and check if retrieved_at column is still NULL or contains a date.

Darkle

clarkwinkelmann You pointed me in the right direction, the issue was with the redis queue, and it was a issue with my environment (a configuration for another site that was preventing the queue from running correctly), so nothing to do with the extension, sorry about that. I left you a message on discord about another matter related to this extension, thank you.

CyberGene

@clarkwinkelmann lately we stopped seeing YouTube previews, only a one liner link is seen. Other websites have a rich embed preview, but not YouTube and I don't see any errors in the logs. What could be the reason?

I haven't changed any settings on my forum for ages. The only recent change is the Flarum security update that I installed a few days ago.

In the Rich Embeds settings I have enabled "Allow playing YouTube videos from the preview". The other two options are disabled ("Use YouTube no-cookie iframe domain", "Enable advanced API-powered rich YouTube previews") and I have no Google API parameters provided.

Also, I've disabled YouTube in the Selective MediaEmbed in order to delegate YouTube rendering to the Rich Embeds.

clarkwinkelmann

CyberGene I don't have a live forum with the extension to check right now, I'll need to take a look on my development computer later.

In the meantime can you share your php flarum info output? It will answer my next question, is FoF Formatting installed?

Is there a YouTube favicon next to the link or nothing at all?

CyberGene

clarkwinkelmann

Flarum core: 1.8.10
PHP version: 8.4.3
MySQL version: 8.0.39-30
Loaded extensions: Core, date, libxml, openssl, pcre, sqlite3, zlib, bcmath, bz2, calendar, ctype, curl, dba, dom, enchant, json, fileinfo, filter, ftp, gd, gettext, gmp, hash, iconv, intl, SPL, mbstring, session, standard, mysqlnd, mysqli, pcntl, PDO, pdo_mysql, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, random, readline, Reflection, exif, shmop, SimpleXML, soap, sockets, sodium, sysvmsg, sysvsem, tidy, tokenizer, xml, xmlreader, xmlwriter, xsl, zip, memcached, Zend OPcache
+-------------------------------------------+---------+--------+
| Flarum Extensions                         |         |        |
+-------------------------------------------+---------+--------+
| ID                                        | Version | Commit |
+-------------------------------------------+---------+--------+
| flarum-flags                              | v1.8.2  |        |
| flarum-approval                           | v1.8.2  |        |
| flarum-tags                               | v1.8.4  |        |
| flarum-suspend                            | v1.8.4  |        |
| flarum-bbcode                             | v1.8.0  |        |
| afrux-forum-widgets-core                  | v0.1.7  |        |
| fof-pages                                 | 1.0.8   |        |
| fof-byobu                                 | 1.4.1   |        |
| flarum-mentions                           | v1.8.5  |        |
| flarum-subscriptions                      | v1.8.1  |        |
| flamarkt-backoffice                       | 0.1.4   |        |
| flarum-markdown                           | v1.8.1  |        |
| zerosonesfun-inline-audio                 | 3.0     |        |
| v17development-seo                        | v2.0.6  |        |
| sycho-move-posts                          | v0.1.8  |        |
| rob006-last-post-avatar                   | 1.2.4   |        |
| kilowhat-rich-embeds                      | 1.3.1   |        |
| ianm-online-guests                        | 1.0.0   |        |
| fof-user-directory                        | 1.4.0   |        |
| fof-user-bio                              | 1.4.2   |        |
| fof-upload                                | 1.8.1   |        |
| fof-sitemap                               | 2.2.1   |        |
| fof-polls                                 | 2.2.12  |        |
| fof-nightmode                             | 1.6.0   |        |
| fof-merge-discussions                     | 1.4.1   |        |
| fof-links                                 | 1.3.0   |        |
| fof-ignore-users                          | 1.2.1   |        |
| fof-forum-statistics-widget               | 1.2.1   |        |
| fof-formatting                            | 1.0.3   |        |
| fof-follow-tags                           | 1.2.9   |        |
| fof-ban-ips                               | 1.1.2   |        |
| fof-anti-spam                             | 1.1.3   |        |
| flarum-sticky                             | v1.8.2  |        |
| flarum-statistics                         | v1.8.1  |        |
| flarum-pusher                             | v1.8.1  |        |
| flarum-nicknames                          | v1.8.2  |        |
| flarum-lock                               | v1.8.2  |        |
| flarum-likes                              | v1.8.1  |        |
| flarum-lang-english                       | v1.8.0  |        |
| flarum-emoji                              | v1.8.1  |        |
| davwheat-session-keepalive                | 1.1.0   |        |
| clarkwinkelmann-selective-mediaembed      | 1.0.1   |        |
| clarkwinkelmann-post-stream-search        | 1.1.0   |        |
| clarkwinkelmann-advanced-search-highlight | 1.0.3   |        |
| blomstra-spam-prevention                  | 1.8.0   |        |
| blomstra-mark-unread                      | 0.2.0   |        |
| blomstra-database-queue                   | 1.1.0   |        |
| askvortsov-rich-text                      | v2.1.7  |        |
| askvortsov-pwa                            | v3.4.1  |        |
| askvortsov-markdown-tables                | v1.2.1  |        |
| afrux-online-users-widget                 | v0.1.9  |        |
+-------------------------------------------+---------+--------+
Base URL: https://myforum.com/forum
Installation path: /home/customer/www/myforum.com/public_html/forum
Queue driver: database
Session driver: file
Scheduler status: Active
Mail driver: smtp
Debug mode: off

clarkwinkelmann is FoF Formatting installed?

Yes, it is. But I have Selective MediaEmbed enabled and YouTube is disabled there to delegate to Rich Embed. That's been a working configuration, I'haven't changed it for years.

clarkwinkelmann Is there a YouTube favicon next to the link or nothing at all?

Yes, the link has the YouTube favicon on the left but that's all there's to it.

« Previous Page Next Page »