Nitro Porter, the inter-software migration toolkit

linc

poVoq There is! Happy to get feedback & questions. It's very basic right now. Hard to see what folks want to know first when you've been the only one coding on it for so long. 😁

Developer Guide for Nitro Porter

seangalla

How about from a Mighty Networks community? 🙂 If so, I'm keen. https://mensgroup.com

linc

seangalla How about from a Mighty Networks community?

I don't know much about it. Can you get a database export from them, or is it API-only?

Yuma

do you have convert ipb 4 for flarum?

luceos

Yuma looks like it: https://github.com/linc/nitro-porter/blob/main/src/Source/IpBoard4.php

linc

Yuma I added preliminary support in Feb but haven't put it in the latest release, yet. It only does the basics so far.

Yuma

linc i have the time ^^

MoshBit

@linc I tried this script today and probably did something wrong because I all I got was a blank screen.

Linode VPS, CentOS v7
nginx 1.23.3
php 8.1.7
MariaDB 10.38
Xenforo 2.2.12

to be honest, I don't know what my PDO Driver is. also I'm willing to donate my database to the tests if you need it, if so, let me know if i should fill out that form?

id love to get my xenforo community ported over to flarum!

MoshBit

So I took another stab at this, and I think where I got confused was that the db prefix field is labeled 'optional'. I didn't put anything in that field and all I got was a blank page with no log file.

I tried again today and put xf_ in the db prefix field and i still got a blank page, but now there's a log file:

[2023-04-08T21:49:20.795134+00:00] porter.INFO: export: User — 14944 rows, 01.16s (6.8mb) [] [] [2023-04-08T21:49:20.803896+00:00] porter.INFO: export: Role — 12 rows, 00.01s (5.8mb) [] [] [2023-04-08T21:49:20.966543+00:00] porter.INFO: export: UserRole — 17437 rows, 00.16s (6.3mb) [] [] [2023-04-08T21:49:21.065402+00:00] porter.INFO: export: UserMeta — 2596 rows, 00.10s (6.5mb) [] [] [2023-04-08T21:49:21.073183+00:00] porter.INFO: export: Category — 70 rows, 00.01s (5.9mb) [] [] [2023-04-08T21:49:24.130848+00:00] porter.INFO: export: Discussion — 25744 rows, 03.06s (9.1mb) [] [] [2023-04-08T21:49:28.965809+00:00] porter.INFO: inserting 'Comment': 100,000 done... [] [] [2023-04-08T21:49:33.870370+00:00] porter.INFO: inserting 'Comment': 200,000 done... [] [] [2023-04-08T21:49:34.706633+00:00] porter.INFO: export: Comment — 216461 rows, 10.58s (7.2mb) [] [] [2023-04-08T21:49:36.160500+00:00] porter.INFO: export: Media — 29331 rows, 01.45s (7mb) [] []

It doesn't say 'success' or anything, but the data appears to be in the db now when I check it with phpmyadmin (in the tables with the PORT_ prefix).

What I don't quite get yet though is what the next step is? This doesn't seem to port the data over to the flaum tables. The user guide says:

Currently, it only uses the default table prefix for targets but you can specify the source prefix.

I'm not sure what the default table prefix for flarum is. All I've been able to find so far is a thread here saying you should avoid using them, so my installation doesn't have db prefixes.

I am using the web installer since I didn't have any luck at all with the CLI. I also have the folder nitro-porter as a sub-directory in my root xenforo install.

linc

MoshBit It doesn't say 'success' or anything, but the data appears to be in the db now when I check it with phpmyadmin (in the tables with the PORT_ prefix).

Ah, that's because it's crashing. It's supposed to do private messages after attachments ("Media"). Sounds like there's a glitch in the Xenforo converter.

linc

@MoshBit Nitro Porter 3.3 should work for you. Just released. The issue was Xenforo allows longer subject lines in PMs than the other systems so they needed to get truncated at 200 characters.

I also fixed a problem where if you were importing to Flarum but didn't preinstall it (in which case Porter will set up the tables), it could lead to a very long delay in the postscript while it was numbering posts due to a missing index.

Some other nice improvements in there too like (very basic) IPB4 support, better logging, an ASP Playground fix, and I took PHPStan up a level (4 to 5). (Most of this work was done over the holidays and just hadn't been released yet).

Jbugy

linc will this work on an old IPB site 3.3.x that runs on cPanel and that would like to keep flarum on the same cPanel (using some modified install)

Thank you

CATER_COM

Hi linc i get some troubles trying to use nitro porter to migrate from Vanilla 3/4 to Flarum. I setup source and destination connections on 2 MySQL databases on 2 différent server.

Edit : Vanilla 3.3 (Garden 3.3 and Conversation 2.4.201) / Flarum latest version from last days

I get a "PDOException SQLSTATE[HY000].... time out" The exception appear after 10 sometimes 20 seconds

Porter's log say
porter.INFO: Permissions are not migrated. Verify all permissions afterward. [] []
porter.INFO: Source: Vanilla 2+ (source) [] []
porter.INFO: Target: Flarum (flarum_test) [] []
porter.INFO: Flag: Use Discussion Body: Enabled [] []

The 2 database are on OVH Private SQL server. The source database in on MySQL 5.6 server and destination is on MySQL 8 server. Each database has its own admin user. The webhosting is shared on OVH (performance) .

Connection settings are the one working on vanilla forum et flarum empty forum.

I tried firts to run Nitro Porter from OVH web root. Thinking the problem would be about time out as explained on your documentation I ran Nitro Porter from an other sharedhost provider that give me more control on php.ini but get the exactly same results. I allowed connection to database from external IP, same results. I don't understand what are those "permisions not migrated". Should I make the migration inside the same database or on the same server with the same user ?
Note that's a little community with less than 1000 discussions and comments..

Thanks for your job with Nitro Porter, this is the ability to keep Vanilla forum content and users that pull me to Flarum.

Cédric

linc

Jbugy It should. If you run into any problems let me know.

pawliniak

Hi, I'm trying to transfer the base from ipboard to flarum, but an error occurs. The database is 600MB and has about 150,000 users. Plesk, PHP 8.1.18. Please help below error.

AH01071: Got error 'PHP message: PHP Fatal error: Uncaught TypeError: fixDuplicateDeletedNames(): Argument #1 ($value) must be of type string, null given in /var/www/vhosts/domain.pl/httpdocs/src/Functions/filter.php:37\nStack trace:\n#0 [internal function]: fixDuplicateDeletedNames()\n#1 /var/www/vhosts/domain.pl/httpdocs/src/Storage.php(88): call_user_func()\n#2 /var/www/vhosts/domain.pl/httpdocs/src/Storage.php(60): Porter\Storage->filterData()\n#3 /var/www/vhosts/domain.pl/httpdocs/src/Storage/Database.php(101): Porter\Storage->normalizeRow()\n#4 /var/www/vhosts/domain.pl/httpdocs/src/ExportModel.php(260): Porter\Storage\Database->store()\n#5 /var/www/vhosts/domain.pl/httpdocs/src/Target/Flarum.php(243): Porter\ExportModel->import()\n#6 /var/www/vhosts/domain.pl/httpdocs/src/Target/Flarum.php(188): Porter\Target\Flarum->users()\n#7 /var/www/vhosts/domain.pl/httpdocs/src/Controller.php(48): Porter\Target\Flarum->run()\n#8 /var/www/vhosts/domai...', referer: https://domain.pl/!<

linc

CATER_COM How many users? That's what gets migrated first.

MoshBit

@linc so I finally got a chance to get a working installation going on my server and test the nitro-porter script, and it did complete once, but I screwed something else up and had to re-do it, and it keeps stalling out on the group_user table:

[2023-05-05T02:41:33.894167+00:00] porter.INFO: Permissions are not migrated. Verify all permissions afterward. [] []
[2023-05-05T02:41:33.894866+00:00] porter.INFO: Source: Xenforo (source) [] []
[2023-05-05T02:41:33.894898+00:00] porter.INFO: Target: Flarum (test) [] []
[2023-05-05T02:41:33.894922+00:00] porter.INFO: Flag: Use Discussion Body: Enabled [] []
[2023-05-05T02:41:35.001690+00:00] porter.INFO: export: User — 15007 rows, 01.11s (7.1mb) [] []
[2023-05-05T02:41:35.009717+00:00] porter.INFO: export: Role — 12 rows, 00.01s (5.9mb) [] []
[2023-05-05T02:41:35.188537+00:00] porter.INFO: export: UserRole — 17508 rows, 00.18s (6.3mb) [] []
[2023-05-05T02:41:35.231811+00:00] porter.INFO: export: UserMeta — 2596 rows, 00.04s (6.5mb) [] []
[2023-05-05T02:41:35.238713+00:00] porter.INFO: export: Category — 70 rows, 00.01s (5.9mb) [] []
[2023-05-05T02:41:38.322973+00:00] porter.INFO: export: Discussion — 25792 rows, 03.08s (9.1mb) [] []
[2023-05-05T02:41:42.961233+00:00] porter.INFO: inserting 'Comment': 100,000 done... [] []
[2023-05-05T02:41:47.719939+00:00] porter.INFO: inserting 'Comment': 200,000 done... [] []
[2023-05-05T02:41:48.549904+00:00] porter.INFO: export: Comment — 216647 rows, 10.23s (7.3mb) [] []
[2023-05-05T02:41:50.008016+00:00] porter.INFO: export: Media — 29361 rows, 01.46s (7.1mb) [] []
[2023-05-05T02:41:52.292850+00:00] porter.INFO: inserting 'Conversation': 100,000 done... [] []
[2023-05-05T02:41:53.093395+00:00] porter.INFO: export: Conversation — 134757 rows, 03.09s (6.5mb) [] []
[2023-05-05T02:41:57.564873+00:00] porter.INFO: inserting 'ConversationMessage': 100,000 done... [] []
[2023-05-05T02:42:04.008870+00:00] porter.INFO: inserting 'ConversationMessage': 200,000 done... [] []
[2023-05-05T02:42:04.448081+00:00] porter.INFO: export: ConversationMessage — 211325 rows, 11.35s (14.3mb) [] []
[2023-05-05T02:42:05.478895+00:00] porter.INFO: inserting 'UserConversation': 100,000 done... [] []
[2023-05-05T02:42:06.391105+00:00] porter.INFO: inserting 'UserConversation': 200,000 done... [] []
[2023-05-05T02:42:07.277068+00:00] porter.INFO: inserting 'UserConversation': 300,000 done... [] []
[2023-05-05T02:42:08.152990+00:00] porter.INFO: inserting 'UserConversation': 400,000 done... [] []
[2023-05-05T02:42:08.263851+00:00] porter.INFO: export: UserConversation — 411608 rows, 03.82s (6.3mb) [] []
[2023-05-05T02:42:09.737557+00:00] porter.INFO: import: users — 15007 rows, 01.38s (7.2mb) [] []
[2023-05-05T02:42:09.806906+00:00] porter.INFO: import: groups — 12 rows, 00.01s (6.2mb) [] []
[2023-05-05T02:42:09.998347+00:00] porter.INFO: import: group_user — 17508 rows, 00.19s (6.6mb) [] []

I feel like I'm close to getting this working, but I'm not sure what to check next (my max memory in PHP is set to like 384mb). Any suggestions?

CATER_COM

MoshBit Oh you also have the "Permissions are not migrated" message but something happen after so I may have some other problem but the log stop after 4 first lines.

linc

MoshBit Usually a stall is a missing index. I'd start with deleting PORT_UserRole and letting Porter recreate it on next run.

linc

CATER_COM "Permissions are not migrated" is not an error, it's just telling you the Porter doesn't do that and never will.

CATER_COM

linc Ok so that mean the process just stop

MoshBit

linc Hmm, is it supposed to start from that table when I re-run it? Because it looks like it's starting from the beginning over again each time, and still getting stuck on group_user after I dropped the PORT_UserRole table.

[2023-05-08T03:13:10.199228+00:00] porter.INFO: Permissions are not migrated. Verify all permissions afterward. [] []
[2023-05-08T03:13:10.200015+00:00] porter.INFO: Source: Xenforo (source) [] []
[2023-05-08T03:13:10.200050+00:00] porter.INFO: Target: Flarum (test) [] []
[2023-05-08T03:13:10.200073+00:00] porter.INFO: Flag: Use Discussion Body: Enabled [] []
[2023-05-08T03:13:11.404780+00:00] porter.INFO: export: User — 15013 rows, 01.20s (7.1mb) [] []
[2023-05-08T03:13:11.410739+00:00] porter.INFO: export: Role — 12 rows, 00.01s (5.9mb) [] []
[2023-05-08T03:13:11.610084+00:00] porter.INFO: export: UserRole — 17514 rows, 00.20s (6.3mb) [] []
[2023-05-08T03:13:11.660788+00:00] porter.INFO: export: UserMeta — 2596 rows, 00.05s (6.5mb) [] []
[2023-05-08T03:13:11.670267+00:00] porter.INFO: export: Category — 70 rows, 00.01s (5.9mb) [] []
[2023-05-08T03:13:15.311471+00:00] porter.INFO: export: Discussion — 25797 rows, 03.64s (9.1mb) [] []
[2023-05-08T03:13:21.091300+00:00] porter.INFO: inserting 'Comment': 100,000 done... [] []
[2023-05-08T03:13:26.718790+00:00] porter.INFO: inserting 'Comment': 200,000 done... [] []
[2023-05-08T03:13:27.595164+00:00] porter.INFO: export: Comment — 216675 rows, 12.28s (7.3mb) [] []
[2023-05-08T03:13:29.066151+00:00] porter.INFO: export: Media — 29364 rows, 01.47s (7.1mb) [] []
[2023-05-08T03:13:31.696713+00:00] porter.INFO: inserting 'Conversation': 100,000 done... [] []
[2023-05-08T03:13:32.597723+00:00] porter.INFO: export: Conversation — 134763 rows, 03.53s (6.5mb) [] []
[2023-05-08T03:13:37.492168+00:00] porter.INFO: inserting 'ConversationMessage': 100,000 done... [] []
[2023-05-08T03:13:44.393002+00:00] porter.INFO: inserting 'ConversationMessage': 200,000 done... [] []
[2023-05-08T03:13:44.867942+00:00] porter.INFO: export: ConversationMessage — 211334 rows, 12.27s (14.3mb) [] []
[2023-05-08T03:13:46.006712+00:00] porter.INFO: inserting 'UserConversation': 100,000 done... [] []
[2023-05-08T03:13:46.977553+00:00] porter.INFO: inserting 'UserConversation': 200,000 done... [] []
[2023-05-08T03:13:47.945350+00:00] porter.INFO: inserting 'UserConversation': 300,000 done... [] []
[2023-05-08T03:13:48.941891+00:00] porter.INFO: inserting 'UserConversation': 400,000 done... [] []
[2023-05-08T03:13:49.106779+00:00] porter.INFO: export: UserConversation — 411627 rows, 04.24s (6.3mb) [] []
[2023-05-08T03:13:50.810270+00:00] porter.INFO: import: users — 15013 rows, 01.59s (10mb) [] []
[2023-05-08T03:13:50.873374+00:00] porter.INFO: import: groups — 12 rows, 00.01s (6.2mb) [] []
[2023-05-08T03:13:51.075927+00:00] porter.INFO: import: group_user — 17514 rows, 00.20s (7.3mb) [] []

I'm running this from the web page, not the CLI just in case that's important. The page goes blank after a while, but it seems like the script keeps going, but I don't know how to check that for sure.

CATER_COM

linc Actually we have 110 users, just few in fact, it's a little regional network. I think the forum size (users and messages) is small enought to not triggerring time out problems. So I understand thta the missing index is on Porter tables, not in Vanilla tables. I 'll try your suggestion.

« Previous Page Next Page »