SMF2 to Flarum migration script - alpha 3

JoshyPHP

The Emoji plugin was only added recently to the Forum bundle, it's possible your version doesn't include it. Bundles are read-only snapshots of a particular configuration. It's not possible to modify them.

However, you can get a configured instance of s9e\TextFormatter\Configurator that you can use to generate your own instance of the parser and render. You can also save them as your own bundle if you want to reuse them more easily. Or you may want to just serialize the parser/renderer and reuse those instead, as Flarum does. That's the preferred method.

$configurator = s9e\TextFormatter\Configurator\Bundles\Forum::getConfigurator();
$configurator->Emoji->addAlias('x.x', '😵');
$configurator->saveBundle('MyBundle', '/tmp/MyBundle.php');

include '/tmp/MyBundle.php';

$text = '[center]x.x[/center]';
$xml  = MyBundle::parse($text);
$html = MyBundle::render($xml);

die("$html\n");

<div style="text-align:center"><img alt="x.x" class="emoji" draggable="false" width="16" height="16" src="//cdn.jsdelivr.net/emojione/assets/png/1f635.png"></div>

marcozambi

JoshyPHP Brilliant! It worked perfectly, thank you very much for your help and for the tool!

doum

I'm waiting for the release of flarum to migrate my SMF2 forum to flarum so please continue your great job ?

I will test it as soon as possible

-ExotiC-

Well done, nice script. In an alternative version, I did not notice any bugs with the display of HTML ? Working perfectly
But i found 1 weirdness - only 61 users was converted from ~502

marcozambi no passwords are exported. Migrated users will have to ask for a new one

This one makes me cry too ?

marcozambi

-ExotiC- But i found 1 weirdness - only 61 users was converted from ~502

Can you see any commonality among those users? That would help a lot to fix a possible bug.
I'm waiting for the first stable to release an updated importer script ?

-ExotiC-

marcozambi
I do not find any common attributes in old SMF database and in a new Flarum db for imported users ?

P.S.: When i run script i have some note for all users, which was founded to export from SMF:

Notice: Undefined offset: 1 in W:\domains\flarum.local\smf2_to_flarum_alternative.php on line 251
User 000018 - lausin2

And after 100 imported discussions i have

Notice: ob_flush(): failed to flush buffer. No buffer to flush in W:\domains\flarum.local\smf2_to_flarum_alternative.php on line 533

How I can rerun it and clear db of flarum for test the script again? Another clean install of flarum without any entry in DB? Or i can just delete all tuples in DB from 'users' table, 'users_discussions', 'user_groups', 'discussions' and 'posts'? And just reopen the script? Thanks.
Sorry for my English.

-ExotiC-

marcozambi I make screenshot of my old SMF database, maybe I missed something.
Orange highlighted users which have been successfully imported to Flarum database.

WARNING! 7013x751 screenshot size ?
http://i.imgur.com/PP0kCYg.jpg

-ExotiC-

@marcozambi So you have any idea what the problem could be? I still can not find a flaw in the code ?

acseven

-ExotiC- @marcozambi
I'm also trying to use this script to test the migration of a SMF board to Flarum and came across this issue (and a bunch of others). Although I can't say why, the user list is being limited by the last part of the query (the WHERE part):

WHERE 
			".$table_prefix."attachments.attachment_type = 0

If you remove that, it will at least select every member instead of just a few (in my case ~100 in a little over 10000).

On other issues:

I don't think anyone has mentioned importing attachments other than images using flagrow/upload, the replacement for flagrow/flarum-ext-image-upload which the script is using. I have no idea on how to approach this, so if anyone has any pointers I'd be welcome to try.
This line makes no sense: $src_dir = $attachments_dir[$idx_src_dir];and I replaced it by $src_dir = $attachments_dir[0]; and along with removing the WHEREstatement I mentioned earlier makes the import output clean and tidy. But making that change leaves me thinking what is the $idx_src_dirvariable for.

That said, so far I've imported Users, Categories (boards/sub-boards) and Posts. I'm going to check on these to see if everything is passing on as it should or not, and later on I'll try the attachments part.

Edit: Posts imported more or less ok, but it timed-out and I'm yet to verify what's missing.

0E800

acseven Is the SMF bbcode, preserved in the posts? Links? Images?

acseven

0E800
Last Edit on: 24th Mar 2017

For organisation purposes, I'm editing this post as I go along using the Alternative version of the script and adding short new posts referring back here:

SCRIPT USAGE RESULTS:

Users:

All Users are imported to the same Member group (group id 3): i.e. you'll have no admins, mods, etc., you'll have to set those manually, and it will be overwritten every time you run the script (unusable password hash included)
Avatars: ?? None of my forum's avatars were imported (does the script process them or not? Also, there are local avatars and external ones!)
User Signatures in Posts: No (missing from flarum?)_

Boards:

Permissions: All of those will be gone, permissions default for all categories

Posts - Text (mostly bbcode):

Post migration: it works, but my database is quite large and the script breaks every time. To be checked later on
Paragraph Text: Mostly Ok, though there is something off with the paragraph spacing, it doesn't seem to be as neat as the one in these posts at this forum. Edit: the script is converting double line break tags   to single line  , making posts less readable (no spacing between paragraphs). Edit #2: it's by default in the script, read mode here
Quotes: Ok as well as Quotes within Quotes
Links: Mostly Ok, BUT the internal URLs are not being corrected. That's to be fixed with rewrites, I guess, but it would be great if the URLs could migrate to the new Flarum URL...
Code: Ok
Bold text: Ok
The script removes all non converted BBcode by default (// Delete all non converted bbcode)

Posts - Media/Other:

Polls: No, polls are not available in flarum, read more here
Embedded Youtube Video URLs: Ok
Images:
- I have yet to run this part of the script, as I would like to try using the flagrow/upload extension instead (to import all attachments, not only images)
- However, images inline to posts are displaying correctly, while linked to their original location/URL
Other Attachments: not possible with the current extension the script supports, needs to be updated to use flagrow/upload

AFTER MIGRATION BOARD FUNCTIONALITY:

Board Permissions: to check
User (Guest) Permissions: to check
Support for Google Analytics
...

INSTALLED EXTENSIONS:

Unrelated to the script, but as I am testing it I should mention I have installed these extensions so far, and they seem to work well:

Flagrow/Ads (github): working
Flagrow/Upload (github): working , though no drag and drop which seems to be a feature
xengine/flarum-ext-markdown-editor or SimpleMDE for Flarum: working, though it would be nice if it would integrate with flagrow/upload
Mediaembed
Flagrow/flarum-ext-image-upload (deprecated, installed just for the SMF-to-Flarum migration script)
Flagrow/bazaar

acseven

One additional thing: the script keeps timing-out when importing Posts. My database is pretty large, so that's the reason, but still there should be a way of knowing what's happening and limiting it.

acseven

I've checked what I mentioned earlier about the paragraphs being off: there is definitely something wrong, as (for example) double break tags (   ) are being imported as only one.

Edit: found it in the code, it's there by definition (but in my case i'll comment it out):
// Replace multiple (one ore more) line breaks with a single one. $text = preg_replace('/[\r\n]{2,}/s', "\n", $text);

Ralkage

The script will need some continuous work when Flarum releases a new build, in which case extensions are inclined to break because of naming convention changes, API changes, etc.

acseven

Ralkage
I wouldn't know how to comment on that with certainty, but I would say that most of what is getting migrated (my log here) is not directly dependant on extensions after migration (maybe the youtube videos?)... In any case, it works pretty well as is, so far, which may be enough for many users.

acseven

One additional information: the only way I apparently would be successful in importing Posts, without timeouts, will by disabling the SQL file dump variable $do_dump = false;. This might have to do with the fact that my server is using nginx and [apparently] if there is no browser output after some time (which happens during the file dump), it will eventually fail with a timeout. I wrote "would be" because the whole table is over 130k messages and the estimated time to complete the process using the script remotely is about 7 hours.
Whenever (if) I decide I can migrate I'll have to figure a better way to do the transfer; I'll probably do it on local DB copies but it would be useful if the script at least had an option to use an existing SQL dump file already on the server, instead of creating one every time.

EvelinaAdamovi

@acseven can you show for us how yours smf2_to_flarum.php file look like? With your changes?

acseven

EvelinaAdamovi

I'm using the alternative version of the script, and here's the changes I'm using - note that I have not tested importing attachments (i.e. images):

acseven Commented this out:

// Replace multiple (one ore more) line breaks with a single one. $text = preg_replace('/[\r\n]{2,}/s', "\n", $text);

acseven Removed this from the User query:

WHERE ".$table_prefix."attachments.attachment_type = 0

acseven

Replaced:

$src_dir = $attachments_dir[$idx_src_dir];

by this

$src_dir = $attachments_dir[0];

EvelinaAdamovi

Hmmm... I'm now thinking what about old urls in forum posts. I mean:

/index.php?topic=1161

now will be not accessible, And truly can't now imagine how solutions may look like for this.

« Previous Page Next Page »