Insert Markdown directly into database (for importer)

wolfman

I am currently writing/extending a migration script for an old SMF-Forum to Flarum. In that process i want to transform some BBCodes to Markdown. Specifically the BBCode for [spoiler] as it gets rendered as HTML <details>-Element, which can be folded out. This is pretty weird if the Spoiler is used inline for a single word or short phrase. Therefore i want the Markdown Spoilers >!spoiler text!<, which work flawlessly inline.

So basically i want to do this in my script and save it to the database:

$str = preg_replace("/\[spoiler\](.*)\[\/spoiler\]/", ">!$1!<", $str);

But this is getting rendered out verbatim as >!some spoiler text!< and not transformed to the black bar over the text which disappears when clicking on it, like: Help me, Obi-Wan Kenobi. You're my only hope.

Even if i look at a "working" spoiler in the database an replace my string like so, it only gets rendered verbatim.

# not working either
$str = preg_replace("/\[spoiler\](.*)\[\/spoiler\]/", "<ISPOILER><s>&gt;!</s>$1<e>!&lt;</e></ISPOILER>", $str);

If i edit a post with the >!some spoiler text!< in the database, the preview immediately renders it correctly as Spoiler. So i guess i have to call some special function for Markdown before i save it to the database, have i? Any hints?

wolfman

I guess i figured it out - or one could say i am an idiot and it was never a real problem! 😑 I already had the necessary TextFormatter with a custom configuration parsing the String that gets saved into the database.

$post = Smf2FlarumFormatterBundle::parse(replaceBodyStrings($str));

In the function replaceBodyStrings() i convert a lot of BBCodes, Smileys, internal Links and other characters and strings that built up over the years. If i replace my Spoiler-BBCodes with either >!$1!< or <ISPOILER><s>>!</s>$1<e>!<</e></ISPOILER> in this function it never gets saved 1:1 into the database, because there is always TextFormatter who parses the String afterwards.

To cirumvent that i disabled/deleted the Parsing of [spoiler] and created a new function, that replaces Strings with Markdown AFTER Textformatter. Hopefully that doesn't mess things up! 😉

$configurator->BBCodes->delete('SPOILER');

...

$post = replaceWithParsedMarkdown(Smf2FlarumFormatterBundle::parse(replaceBodyStrings($str)));

...

/**
 * Utility function to replace some characters with Markdown before storing in Flarum
 * This must be called AFTER TextFormatter::parse() or Smf2FlarumFormatterBundle::parse() respectively
 */
function replaceWithParsedMarkdown($str)
{
    // Replace [spoiler] with Markdowns inline spoiler (and not as HTML-details element)
    $str = preg_replace("/\[spoiler\](.*)\[\/spoiler\]/", "<ISPOILER><s>&gt;!</s>$1<e>!&lt;</e></ISPOILER>", $str);

    return $str;
}

Edit: Forgot the disabling of BBCode-Parsing

clarkwinkelmann

As long as you are inserting valid XML in the content column of the database, Flarum shouldn't break.

It's difficult to tell what might be going wrong without seeing what exactly goes on in Smf2FlarumFormatterBundle::parse. Since you shared a line that calls $configurator->BBCodes I assume you are using the actual TextFormatter package.

If you enable the same options and plugins as Flarum in the TextFormatter library, you should get the same XML output that Flarum would generate.

Some of Flarum extensions might define additional or modified behavior for TextFormatter that would have to be copied as well, but I don't think spoilers have anything special, we are using the tag that's included in TextFormatter.

wolfman

Yes, i am using TextFormatter (https://github.com/wolfgang-demeter/smf2flarum/blob/master/migrate.php#L33) to add some custom BBCodes - this is also necessary for the migrated attachments to FoF Upload.
The tests so far look good! 😀