Nitro Porter, the inter-software migration toolkit

schwabing

linc what a great initiative!
We have a rather large forum, custom built, but straightforward with only very few features. 3GB of SQL, roughly 6 million posts (not threads), 700-800 concurrent users online during evening hours, 1 million monthly sessions. I would love to move it to flarum.
As I read you're pretty busy right now... Is there maybe someone you can recommend for a paid migration?
If not what would be the best approach? Learn how Nitro Porter works from the other sources and write one for our case?
https://github.com/linc/nitro-porter/blob/089091256f56778dabdc46d7b3fbe52b39320a16/src/Source/Xenforo.php

luceos

schwabing there are some partners listed here that can help with a migration: https://flarum.org/partners, in addition you can post under services to get potential contractors to respond to your request.

linc

schwabing Is there maybe someone you can recommend for a paid migration?

Yup, @luceos has you covered. I do think you could use the existing src/Source/* packages as a model and probably move a basic set of discussions, posts, users, and categories over fairly quickly if you have a PHP engineer handy. It's all the other stuff that usually gets more complicated. Unfortuantely I'm not able to assist with custom / bespoke forums, there's just too damn many even limiting it to the distributed ones. 😅

poVoq

linc Is there a guide for adding additional export targets? I think Lemmy might be really interesting for that.

linc

poVoq There is! Happy to get feedback & questions. It's very basic right now. Hard to see what folks want to know first when you've been the only one coding on it for so long. 😁

Developer Guide for Nitro Porter

Yuma

linc it's not easy for my database ipb4 i have see it's great job to convert all correctly for flarum : )

seangalla

How about from a Mighty Networks community? 🙂 If so, I'm keen. https://mensgroup.com

linc

seangalla How about from a Mighty Networks community?

I don't know much about it. Can you get a database export from them, or is it API-only?

Yuma

do you have convert ipb 4 for flarum?

luceos

Yuma looks like it: https://github.com/linc/nitro-porter/blob/main/src/Source/IpBoard4.php

linc

Yuma I added preliminary support in Feb but haven't put it in the latest release, yet. It only does the basics so far.

Yuma

linc i have the time ^^

MoshBit

@linc I tried this script today and probably did something wrong because I all I got was a blank screen.

Linode VPS, CentOS v7
nginx 1.23.3
php 8.1.7
MariaDB 10.38
Xenforo 2.2.12

to be honest, I don't know what my PDO Driver is. also I'm willing to donate my database to the tests if you need it, if so, let me know if i should fill out that form?

id love to get my xenforo community ported over to flarum!

MoshBit

So I took another stab at this, and I think where I got confused was that the db prefix field is labeled 'optional'. I didn't put anything in that field and all I got was a blank page with no log file.

I tried again today and put xf_ in the db prefix field and i still got a blank page, but now there's a log file:

[2023-04-08T21:49:20.795134+00:00] porter.INFO: export: User — 14944 rows, 01.16s (6.8mb) [] [] [2023-04-08T21:49:20.803896+00:00] porter.INFO: export: Role — 12 rows, 00.01s (5.8mb) [] [] [2023-04-08T21:49:20.966543+00:00] porter.INFO: export: UserRole — 17437 rows, 00.16s (6.3mb) [] [] [2023-04-08T21:49:21.065402+00:00] porter.INFO: export: UserMeta — 2596 rows, 00.10s (6.5mb) [] [] [2023-04-08T21:49:21.073183+00:00] porter.INFO: export: Category — 70 rows, 00.01s (5.9mb) [] [] [2023-04-08T21:49:24.130848+00:00] porter.INFO: export: Discussion — 25744 rows, 03.06s (9.1mb) [] [] [2023-04-08T21:49:28.965809+00:00] porter.INFO: inserting 'Comment': 100,000 done... [] [] [2023-04-08T21:49:33.870370+00:00] porter.INFO: inserting 'Comment': 200,000 done... [] [] [2023-04-08T21:49:34.706633+00:00] porter.INFO: export: Comment — 216461 rows, 10.58s (7.2mb) [] [] [2023-04-08T21:49:36.160500+00:00] porter.INFO: export: Media — 29331 rows, 01.45s (7mb) [] []

It doesn't say 'success' or anything, but the data appears to be in the db now when I check it with phpmyadmin (in the tables with the PORT_ prefix).

What I don't quite get yet though is what the next step is? This doesn't seem to port the data over to the flaum tables. The user guide says:

Currently, it only uses the default table prefix for targets but you can specify the source prefix.

I'm not sure what the default table prefix for flarum is. All I've been able to find so far is a thread here saying you should avoid using them, so my installation doesn't have db prefixes.

I am using the web installer since I didn't have any luck at all with the CLI. I also have the folder nitro-porter as a sub-directory in my root xenforo install.

linc

MoshBit It doesn't say 'success' or anything, but the data appears to be in the db now when I check it with phpmyadmin (in the tables with the PORT_ prefix).

Ah, that's because it's crashing. It's supposed to do private messages after attachments ("Media"). Sounds like there's a glitch in the Xenforo converter.

linc

@MoshBit Nitro Porter 3.3 should work for you. Just released. The issue was Xenforo allows longer subject lines in PMs than the other systems so they needed to get truncated at 200 characters.

I also fixed a problem where if you were importing to Flarum but didn't preinstall it (in which case Porter will set up the tables), it could lead to a very long delay in the postscript while it was numbering posts due to a missing index.

Some other nice improvements in there too like (very basic) IPB4 support, better logging, an ASP Playground fix, and I took PHPStan up a level (4 to 5). (Most of this work was done over the holidays and just hadn't been released yet).

Jbugy

linc will this work on an old IPB site 3.3.x that runs on cPanel and that would like to keep flarum on the same cPanel (using some modified install)

Thank you

CATER_COM

Hi linc i get some troubles trying to use nitro porter to migrate from Vanilla 3/4 to Flarum. I setup source and destination connections on 2 MySQL databases on 2 différent server.

Edit : Vanilla 3.3 (Garden 3.3 and Conversation 2.4.201) / Flarum latest version from last days

I get a "PDOException SQLSTATE[HY000].... time out" The exception appear after 10 sometimes 20 seconds

Porter's log say
porter.INFO: Permissions are not migrated. Verify all permissions afterward. [] []
porter.INFO: Source: Vanilla 2+ (source) [] []
porter.INFO: Target: Flarum (flarum_test) [] []
porter.INFO: Flag: Use Discussion Body: Enabled [] []

The 2 database are on OVH Private SQL server. The source database in on MySQL 5.6 server and destination is on MySQL 8 server. Each database has its own admin user. The webhosting is shared on OVH (performance) .

Connection settings are the one working on vanilla forum et flarum empty forum.

I tried firts to run Nitro Porter from OVH web root. Thinking the problem would be about time out as explained on your documentation I ran Nitro Porter from an other sharedhost provider that give me more control on php.ini but get the exactly same results. I allowed connection to database from external IP, same results. I don't understand what are those "permisions not migrated". Should I make the migration inside the same database or on the same server with the same user ?
Note that's a little community with less than 1000 discussions and comments..

Thanks for your job with Nitro Porter, this is the ability to keep Vanilla forum content and users that pull me to Flarum.

Cédric

linc

Jbugy It should. If you run into any problems let me know.

pawliniak

Hi, I'm trying to transfer the base from ipboard to flarum, but an error occurs. The database is 600MB and has about 150,000 users. Plesk, PHP 8.1.18. Please help below error.

AH01071: Got error 'PHP message: PHP Fatal error: Uncaught TypeError: fixDuplicateDeletedNames(): Argument #1 ($value) must be of type string, null given in /var/www/vhosts/domain.pl/httpdocs/src/Functions/filter.php:37\nStack trace:\n#0 [internal function]: fixDuplicateDeletedNames()\n#1 /var/www/vhosts/domain.pl/httpdocs/src/Storage.php(88): call_user_func()\n#2 /var/www/vhosts/domain.pl/httpdocs/src/Storage.php(60): Porter\Storage->filterData()\n#3 /var/www/vhosts/domain.pl/httpdocs/src/Storage/Database.php(101): Porter\Storage->normalizeRow()\n#4 /var/www/vhosts/domain.pl/httpdocs/src/ExportModel.php(260): Porter\Storage\Database->store()\n#5 /var/www/vhosts/domain.pl/httpdocs/src/Target/Flarum.php(243): Porter\ExportModel->import()\n#6 /var/www/vhosts/domain.pl/httpdocs/src/Target/Flarum.php(188): Porter\Target\Flarum->users()\n#7 /var/www/vhosts/domain.pl/httpdocs/src/Controller.php(48): Porter\Target\Flarum->run()\n#8 /var/www/vhosts/domai...', referer: https://domain.pl/!<

linc

CATER_COM How many users? That's what gets migrated first.

MoshBit

@linc so I finally got a chance to get a working installation going on my server and test the nitro-porter script, and it did complete once, but I screwed something else up and had to re-do it, and it keeps stalling out on the group_user table:

[2023-05-05T02:41:33.894167+00:00] porter.INFO: Permissions are not migrated. Verify all permissions afterward. [] []
[2023-05-05T02:41:33.894866+00:00] porter.INFO: Source: Xenforo (source) [] []
[2023-05-05T02:41:33.894898+00:00] porter.INFO: Target: Flarum (test) [] []
[2023-05-05T02:41:33.894922+00:00] porter.INFO: Flag: Use Discussion Body: Enabled [] []
[2023-05-05T02:41:35.001690+00:00] porter.INFO: export: User — 15007 rows, 01.11s (7.1mb) [] []
[2023-05-05T02:41:35.009717+00:00] porter.INFO: export: Role — 12 rows, 00.01s (5.9mb) [] []
[2023-05-05T02:41:35.188537+00:00] porter.INFO: export: UserRole — 17508 rows, 00.18s (6.3mb) [] []
[2023-05-05T02:41:35.231811+00:00] porter.INFO: export: UserMeta — 2596 rows, 00.04s (6.5mb) [] []
[2023-05-05T02:41:35.238713+00:00] porter.INFO: export: Category — 70 rows, 00.01s (5.9mb) [] []
[2023-05-05T02:41:38.322973+00:00] porter.INFO: export: Discussion — 25792 rows, 03.08s (9.1mb) [] []
[2023-05-05T02:41:42.961233+00:00] porter.INFO: inserting 'Comment': 100,000 done... [] []
[2023-05-05T02:41:47.719939+00:00] porter.INFO: inserting 'Comment': 200,000 done... [] []
[2023-05-05T02:41:48.549904+00:00] porter.INFO: export: Comment — 216647 rows, 10.23s (7.3mb) [] []
[2023-05-05T02:41:50.008016+00:00] porter.INFO: export: Media — 29361 rows, 01.46s (7.1mb) [] []
[2023-05-05T02:41:52.292850+00:00] porter.INFO: inserting 'Conversation': 100,000 done... [] []
[2023-05-05T02:41:53.093395+00:00] porter.INFO: export: Conversation — 134757 rows, 03.09s (6.5mb) [] []
[2023-05-05T02:41:57.564873+00:00] porter.INFO: inserting 'ConversationMessage': 100,000 done... [] []
[2023-05-05T02:42:04.008870+00:00] porter.INFO: inserting 'ConversationMessage': 200,000 done... [] []
[2023-05-05T02:42:04.448081+00:00] porter.INFO: export: ConversationMessage — 211325 rows, 11.35s (14.3mb) [] []
[2023-05-05T02:42:05.478895+00:00] porter.INFO: inserting 'UserConversation': 100,000 done... [] []
[2023-05-05T02:42:06.391105+00:00] porter.INFO: inserting 'UserConversation': 200,000 done... [] []
[2023-05-05T02:42:07.277068+00:00] porter.INFO: inserting 'UserConversation': 300,000 done... [] []
[2023-05-05T02:42:08.152990+00:00] porter.INFO: inserting 'UserConversation': 400,000 done... [] []
[2023-05-05T02:42:08.263851+00:00] porter.INFO: export: UserConversation — 411608 rows, 03.82s (6.3mb) [] []
[2023-05-05T02:42:09.737557+00:00] porter.INFO: import: users — 15007 rows, 01.38s (7.2mb) [] []
[2023-05-05T02:42:09.806906+00:00] porter.INFO: import: groups — 12 rows, 00.01s (6.2mb) [] []
[2023-05-05T02:42:09.998347+00:00] porter.INFO: import: group_user — 17508 rows, 00.19s (6.6mb) [] []

I feel like I'm close to getting this working, but I'm not sure what to check next (my max memory in PHP is set to like 384mb). Any suggestions?

CATER_COM

MoshBit Oh you also have the "Permissions are not migrated" message but something happen after so I may have some other problem but the log stop after 4 first lines.

linc

MoshBit Usually a stall is a missing index. I'd start with deleting PORT_UserRole and letting Porter recreate it on next run.

« Previous Page Next Page »