FoF Sitemap

DursunCan

IanM I used the new command. The sitemap.xml file was not created. You can see the command output in my previous answer.

luceos

DursunCan go to your admin area set the mode to cache to disk or whatever it is named. You have the cache option selected which means the file is generated by php on the fly when you hit the URL.

rafaucau

I'm trying to build a sitemap for a forum that has 400k discussions and 5 million posts and it throws this error:

FoF Sitemap: running in multi-file mode
PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 255

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 255

rafaucau

My hosting support said I need to run the command like this: php -d memory_limit=-1 flarum fof:sitemap:multi and it seems to be working - it doesn't throw an error and it seems to be generating.

If the sitemap is generated, do I then no longer need to run the php flarum fof:sitemap:multi command and cron (php flarum schedule:run) will automatically update it?

luceos

rafaucau I assume your cron will also run out of memory unless you also prefix it with that php -d memory_limit=-1. From what I remember if you selected the multi mode in admin it should then auto create the required files yes.

rafaucau

luceos The site map has been generated, but when I try to go to its url, it loads endlessly and I get a timeout

luceos

rafaucau what mode is enabled in your admin area?

rafaucau

luceos Multi file

rafaucau

luceos I got this response from my hosting support (translated):

The /public/sitemaps directory contains the sitemaps files, which have the .xml.gz format, although the /storage/sitemaps-processing directory shows the sitemap.xml file.

When it is placed in the /public directory it is loaded correctly. It appears that the location of the sitemap.xml file is incorrect and it is placed in a different directory than the main public one.

When the sitemap generates on cron, this manually moved "sitemap.xml" file disappears.

rafaucau

No one can help? :<

clarkwinkelmann

rafaucau it might be an instance of this reported issue FriendsOfFlarum/sitemap28

Unfortunately nobody from the team has been able to take a look at that issue yet.

rafaucau

clarkwinkelmann We sent a donate for you (fof) to buy a beer and possibly think about the problem 😄

pkernstock

Is it expected that sitemap.xml exposes all user accounts in the sitemap.xml?

<url>
<loc>https://domain.tld/u/userX</loc>
<lastmod>2021-06-26T00:28:29+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>0.5</priority>
</url>
<url>
<loc>https://domain.tld/u/userX</loc>
<lastmod>2021-06-26T00:28:29+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>0.5</priority>
</url>

Might be a side-effect of an other extension, but I guess it would make sense to check if the entry is an actual post or a user?

clarkwinkelmann

pkernstock it's aligned with the "Search users" permission.

The Sitemap extension retrieves all the users that a guest visitor can see via the Flarum REST API.

Hari

pkernstock for safety reasons disallow robots to /u/ something similar happend to me too

pkernstock

clarkwinkelmann That's strange, as those permission is se to "Members" in my case. But still, is there indeed a scenario where all user accounts (and usernames) should be exposed via sitemap.xml? Specially in large forums this causes a really, really large sitemap.xml file (without any benefits for search engines or so)

clarkwinkelmann

pkernstock this is how it's currently implemented. I don't think the user source can be disabled with a user setting at the moment.

If the users show up when "Search Users" is set to members there must be an issue, that's not normal. Maybe the feature was broken by the recent changes that renamed "View User List" to "Search Users".

luceos

clarkwinkelmann this is indeed very much a bug then.

IanM

1.0.1

Use slug drivers correctly FriendsOfFlarum/sitemap30 (thanks @askvortsov )

Hari

IanM it's been almost 4 hours extiverse.com is not picking up this update. 🤔

« Previous Page Next Page »