Hari Hey, glad you found the solution! Can you explain why should I disable the setting for "Allow all bots & crawl full site directory" in the SEO ?

  • Hari replied to this.

      girefko i could not recall it, when i enable it i think the site directory setting included user directory from fof extension (not sure)

      I thought indexing discussions is good enough and there is one more setting called crwal full discussion it basically enables spiders to go through entire discussion which causes huge load on server in case of larger communities.

        a month later

        Mercury shows there’s a new version 2.0.0-beta.1. I guess that’s a mistake. Or can we update to it?

        • IanM replied to this.

          CyberGene We are currently working on a rewrite of the extension to better support large communities with discussions and/or users in the millions.

          2.0.0-beta.1 is our first iteration, which works fairly well, but still has some performance issues when the counts are in the multi-million range.

          For the time being, I'd suggest not upgrading to this, unless you have a specific need to do so. Once we've ironed out all of the issues, we will remove the beta label and publish the new version

              IanM We are currently working on a rewrite of the extension to better support large communities with discussions and/or users in the millions.

              Not that my flarum is that big yet but excited to hear about the continued development of this extension!

                a month later

                Sitemap version 2.0

                • PHP 8.0 requirement
                • Complete rewrite of the deploy system.
                • Simplification of the available modes: now only runtime and cached multi-file are available.
                • Runtime now depends on the same logic as multi-file, so it is no longer limited to 50.000 items. But it's still not advisable to use it with so many items.
                • Added admin setting to exclude all user profiles from sitemap.
                • Added ForceCached extender.
                • Added optional "risky" performance improvements. This option is likely not for you. Consult the README for details.

                To update from version 1.x

                The update requires PHP 8.0 or greater. If you are still on PHP 7.3 or 7.4, you can continue using version 1.x. There will be no new updates for the 1.x line.

                If composer update doesn't pick up the new version, you can force the update to version 2.0 with this command:

                composer require fof/sitemap:"*"

                Clear the cache:

                php flarum cache:clear

                If you were running runtime mode, that's everything that was needed. The extension should work right away.

                If you were running cache or disk mode, the mode will automatically switch to the new multi-file mode.

                Then run the following command to generate a first sitemap immediately:

                php flarum fof:sitemap:build

                If you had already configured the scheduler, the new build command will automatically run at the previously selected time.

                If you get a filesystem write error, verify Flarum can write to the public folder. A sitemaps folder will be created and all subsequent write operations will be in that folder.

                This update was sponsored by Blomstra.

                  i have tried these codes, but did not work

                  composer update fof/sitemap
                  php flarum migrate
                  php flarum cache:clear

                  and

                  composer require fof/sitemap:"*"

                    meihuak is there an error? Can you share your php flarum info output and also the full composer output of the require command?

                        clarkwinkelmann thanks for this great update. Went smoothly on two PHP 8.1 Flarum installations. Needed the composer require fof/sitemap:"*" command though.
                        Finnish translation coming later today since there was one key change in the translations 50000->10000.

                          clarkwinkelmann
                          thanks, i found the problem, my PHP version is 7.3.

                          i have updated the PHP to 8.1, after i run the upload code, there are some yellow warning, but finally the extension has been updated to 2.0.

                          is there some error?

                          `PHP Deprecated: Return type of Symfony\Component\Console\Helper\HelperSet::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/console/Helper/HelperSet.php on line 112

                          Deprecated: Return type of Symfony\Component\Console\Helper\HelperSet::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/console/Helper/HelperSet.php on line 112
                          Deprecation Notice: strlen(): Passing null to parameter #1 ($string) of type string is deprecated in phar:///usr/sbin/composer/vendor/justinrainbow/json-schema/src/JsonSchema/Constraints/Constraint.php:48
                          Deprecation Notice: Return type of Composer\Repository\ArrayRepository::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/Repository/ArrayRepository.php:286
                          Deprecation Notice: Return type of Composer\Repository\ArrayRepository::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/Repository/ArrayRepository.php:286
                          Deprecation Notice: Return type of Composer\Repository\ArrayRepository::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/Repository/ArrayRepository.php:286
                          Loading composer repositories with package information
                          Info from https://repo.packagist.org: #StandWithUkraine
                          Deprecation Notice: Return type of Composer\DependencyResolver\Pool::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Pool.php:82
                          Updating dependencies
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSet::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSet.php:119
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSet::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSet.php:101
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:155
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:165
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:160
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:170
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:150
                          Deprecation Notice: Return type of Composer\DependencyResolver\Decisions::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/Decisions.php:145
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSetIterator::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSetIterator.php:36
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSetIterator::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSetIterator.php:46
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSetIterator::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSetIterator.php:41
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSetIterator::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSetIterator.php:89
                          Deprecation Notice: Return type of Composer\DependencyResolver\RuleSetIterator::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/DependencyResolver/RuleSetIterator.php:70
                          Lock file operations: 0 installs, 1 update, 0 removals

                          • Upgrading fof/sitemap (1.0.3 => 2.0.0)
                            Writing lock file
                            Installing dependencies from lock file (including require-dev)
                            Package operations: 0 installs, 1 update, 0 removals
                          • Downloading fof/sitemap (2.0.0)
                          • Upgrading fof/sitemap (1.0.3 => 2.0.0): Extracting archive
                            Deprecation Notice: Return type of Symfony\Component\Finder\Finder::getIterator() should either be compatible with IteratorAggregate::getIterator(): Traversable, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Finder.php:675
                            Deprecation Notice: Return type of Symfony\Component\Finder\Finder::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Finder.php:732
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\RecursiveDirectoryIterator::getChildren() should either be compatible with RecursiveDirectoryIterator::getChildren(): RecursiveDirectoryIterator, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php:85
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\RecursiveDirectoryIterator::rewind() should either be compatible with FilesystemIterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php:113
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\RecursiveDirectoryIterator::current() should either be compatible with FilesystemIterator::current(): SplFileInfo|FilesystemIterator|string, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/RecursiveDirectoryIterator.php:65
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\FilterIterator::rewind() should either be compatible with FilterIterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/FilterIterator.php:30
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\DepthRangeFilterIterator::accept() should either be compatible with FilterIterator::accept(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/DepthRangeFilterIterator.php:41
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\FilenameFilterIterator::accept() should either be compatible with FilterIterator::accept(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/FilenameFilterIterator.php:28
                            Deprecation Notice: Return type of Composer\Repository\CompositeRepository::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/src/Composer/Repository/CompositeRepository.php:180
                            Package swiftmailer/swiftmailer is abandoned, you should avoid using it. Use symfony/mailer instead.
                            Generating autoload files
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\FileTypeFilterIterator::accept() should either be compatible with FilterIterator::accept(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/FileTypeFilterIterator.php:42
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\ExcludeDirectoryFilterIterator::accept() should either be compatible with FilterIterator::accept(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/ExcludeDirectoryFilterIterator.php:55
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\ExcludeDirectoryFilterIterator::hasChildren() should either be compatible with RecursiveIterator::hasChildren(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/ExcludeDirectoryFilterIterator.php:71
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\ExcludeDirectoryFilterIterator::getChildren() should either be compatible with RecursiveIterator::getChildren(): ?RecursiveIterator, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/ExcludeDirectoryFilterIterator.php:76
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\FilterIterator::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/FilterIterator.php:30
                            Deprecation Notice: Return type of Symfony\Component\Finder\Iterator\PathFilterIterator::accept() should either be compatible with FilterIterator::accept(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in phar:///usr/sbin/composer/vendor/symfony/finder/Iterator/PathFilterIterator.php:27
                            77 packages you are using are looking for funding.
                            Use the composer fund command to find out more!`

                              luceos hey thanks, i dont know technology, i just want to make sure if i updated the extension successfully and my flarum will work correctly.

                                Something is wrong. I updated without any problems, cleared the cache. Previously the sitemap for my forum was here:
                                https://pianoclack.com/forum/sitemap.xml

                                However when I open this address I get redirected to another address where the sitemap is empty:
                                https://pianoclack.com/forum/sitemap-live/index

                                P.S. I rolled back already, so the first URL should be working OK because I'm on 1.0.2

                                What is wrong? How can I uninstall and return to previous since this has the potential of messing the Google index.

                                  I hope this gets sorted quickly. I'm wondering if there's an easy way to downgrade though until that happens.

                                  OK, whoever needs to downgrade to the last known stable version:
                                  composer require fof/sitemap:1.0.2
                                  php flarum cache:clear

                                  clarkwinkelmann Added admin setting to exclude all user profiles from sitemap.

                                  This setting is missing!

                                  Solved:

                                  My bad I forgot to remove the below code from extend.php 😃 thanks for adding this feature to the extension.

                                  (new \FoF\Sitemap\Extend\RemoveResource(\FoF\Sitemap\Resources\User::class)),

                                      Hari This setting is missing!

                                      It will appear if you remove the custom setting in extend.php that was there for the old version 🙂 While I was testing the new version, I thought the setting was for the old version and commented it out in the extend.php then it appeared in the settings.