Good news, let me see if finally i can delete my pure PHP script. Soon as possible i will return with some feedback ?

This plugin is a must in 2018.

Feature-wise it's identical to this other sitemap extension, the main difference being that this one doesn't write anything to the filesystem. That's an advantage because this way you don't have to deal with annoying permissions issues on the filesystem.

I tried to go with an existing sitemap library, but as it turns out they are either very outdated and not published to Packagist or very recent but too tightly integrated with Laravel. In the end I re-implemented the logic by copying small bits from those libraries, which wasn't that hard to do.

I first tried to cache the sitemap output to replicate the file-on-disk behavior, but after some thoughts I decided it wasn't worth it. The queries to generate the sitemap aren't that complicated. More complicated queries are run everytime you load the homepage. So I'm just returning a fresh and up to date sitemap every time you ask for it.

If there's interest I can implement the links limit (number and file size) according to the sitemap specs. The specs says there shouldn't be more than 50k links per sitemap file. For now it will keep growing forever. I'm not sure what is the actual search engine behavior if you exceed 50k links per file.

Other features that could be added are the ability to select what type of page is added to the sitemap, an option to choose the sitemap index file name/url and an option to cache some of the output for a period of time.

Don't hesitate to open an issue on the repo with your wanted feature and use case if you want to discuss a suggestion !

    clarkwinkelmann Awesome work!

    Maybe for the sitemap limit allow us to have a different sitemap for each of our tags?

      About don't write sitemap to the disk is a good idea, i love it. Thinking about big sites, add the file to the cache is the way to go... because query can be simple, but big sites can output easily more than 5 MB of data with a sitemap.

      The feature that i miss a lot is have more control over parts of the board added to the sitemap (users, discussions and tags). Will be nice have a an option to include/exclude these three parts of the forum. Imagine that i decide not index tags, at this momet the plugin is limited because if i added a robots.txt to not allow indexing tags but i'm sending a sitemap that contain tag's urls, this is a confusing signal to the search engines.

      Also be able to setup the "changefreq" and "priority" parameters according to the forum part will be great.

      Also observed that:

      • On title changes the lastmod is not changed ❌
      • On reply added the lastmod is changed ✅
      • On edit contents of a post into discussion, the lastmod is not changed ❌

      The Google's behavior with sitemaps bigger than 50k urls or 10MB is simply don't process them.

      Hope that post this here also works instead go to GitHub ?

      16 days later
      a month later

      tlalok is it Flarum's 404 page (with the error pages extension) or the webserver 404 page ?

      If it's the webserver error page, make sure the /sitemap.xml url is handled by Flarum's index.php via htaccess or rewrite rules (should be with provided htaccess and example config). Maybe there are special rules in some hostings that prevent this file from being dynamically generated.

      If it's Flarum 404 page, the extension is likely not enabled correctly.

      You may open an issue at https://github.com/flagrow/sitemap/issues if you need further help ?

          tlalok Try disabling & re-enabling the extension.

            clarkwinkelmann Indeed, I just added this nginx directive and it works.

            location = /sitemap.xml { try_files $uri $uri/ /index.php?$query_string; }

            clarkwinkelmann If it's the webserver error page, make sure the /sitemap.xml url is handled by Flarum's index.php via htaccess or rewrite rules (should be with provided htaccess and example config). Maybe there are special rules in some hostings that prevent this file from being dynamically generated.

            I use this docker image, so I am going to fix this issue.

            @clarkwinkelmann @arda thanks for your help ?

                  2 months later
                  8 days later

                  Updated for beta 8.

                  I've kept compatibility with the Pages extension for now, even though it hasn't been updated for beta 8 yet. Be warned that an additional update of this extension might be needed to support the future beta8-compatible version of Pages.

                    clarkwinkelmann Thanks for the update, unfortunately it doesn't seem to work at my end. When accessing url/sitemap.xml I'm only getting a 404, both with the extension enabled and disabled. I'm using nginx as my webserver.

                        Kakifrucht Could this be a similar case to tlalok where the webserver needs an additional rewrite rule to handle .xml files via the PHP application ? Maybe try the solution above to see if it solves your case.

                        Also is the 404 from Flarum (there should be a back to homepage link) or Nginx (just a 404 message in english) ? If nginx is responding, then it has something to do with server configuration (and it seems the fix above solve it).

                        If it's a Flarum 404 error page then we'll have to investigate further. Could you open an issue at GitHub if that's the case so we don't create too much noise here ?

                              clarkwinkelmann That's what I was looking for, sorry for being too lazy to scroll a couple of posts up.

                              You might want to add it to the installation section of this discussion for nginx users.

                                  Kakifrucht You might want to add it to the installation section of this discussion for nginx users.

                                  Yes indeed, I'll do that. Though it doesn't seem to apply to everybody. I didn't get any feedback of anybody getting it working on nginx, but I'm running the extension on my own nginx server and didn't need the additional config.

                                  I should try to narrow it down to identify what's the cause for this, but I don't have the time 💩

                                      clarkwinkelmann but I'm running the extension on my own nginx server and didn't need the additional config.

                                      I really only followed the install instructions for nginx from here.

                                      This is what my server configuration block looks like:

                                      server {
    listen 80;
    server_name myurl.com;
    root path/to/flarum/public;
    include path/to/flarum/.nginx.conf;
    
    location = /sitemap.xml {
        try_files $uri $uri/ /index.php?$query_string;
    }
    
    location ~ \.php$ {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/run/php/php7.2-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
    }
}

                                      I then used certbot to apply a SSL certificate and set it to redirect everything to https.

                                        3 months later

                                        • [deleted]

                                        • Post #24
                                        • Edited

                                        FriendsOfFlarum How can i remove users in sitemap.

                                        <loc>https://mysite.com/u/user1</loc>
<lastmod>2019-03-25T18:04:08+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>0.5</priority>
<url>
<loc>https://mysite.com/u/user2</loc>
<lastmod>2019-03-25T18:04:08+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>0.5</priority>

                                        Solved: Open SitemapGenerator.php in vendor/flagrow/sitemap/src

                                        Remove lines:

                                        $users = User::whereVisibleTo(new Guest())->get();

    foreach ($users as $user) {
        $urlSet->addUrl($url . '/u/' . $user->username, Carbon::now(), Frequency::DAILY, 0.5);
    }

                                            [deleted] this should only be considered a temporary solution because next time you run composer those files might be reverted to their original state.

                                            The sitemap extension is permission based, so if you disable public user listing via Flarum permissions, they should also disapear from the sitemap.

                                            There is no manual options at this time. The idea is that any publicly visible page should be in the sitemap.