Dofollow letting the bot crawl the target page, when user replies to the post and when see the HTML source code why flarum is marking its own domain links to NoFollow? our own website links always should be dofollow

this is something that needs to be fixed in the core because it is coming from the core.

https://moz.com/community/q/topic/32531/dofollow-and-nofollow-links
https://blog.nextsale.io/dofollow-or-nofollow-links-internal-linking-seo/

there is this extension not sure about its application/use case Internal Link Dofollow

If something is done by core it should be done correctly otherwise core should not add this!

    Flarum currently doesn't make any distinction between the types of links in a post, all are considered "user generated content", and as such have the nofollow tag added to them. So it's not really a bug, just a missing feature.

    It would probably be useful to other extensions outside of the rel attribute if Flarum could also add a special attribute/class to those links to mark them as internal links.

    This also ties in with the fact Flarum doesn't route those internal links inside of the single page app. If we implement a way for Flarum to recognize internal links, we should also make them route the SPA without refreshing the page (I believe an extension was created to address this particular issue in the past)

    Hari otherwise core should not add this

    The reason this is the built-in default is because it's a recommended best practice to apply nofollow on user generated content, this prevents the feature being abused to promote external websites by spamming links. Until internal link recognition is implemented in Flarum, this is the safer default to apply to all links.

    There's another aspect to this, just recognizing internal links based on domain/subfolder might not be enough. Removing nofollow from dead internal links could cause SEO downsides as well. Bad actors could spam unique 404 links, causing search engines to be slowed down and taking longer to index useful pages. This part is definitely too difficult to be handled in core. All internal links in a post should be checked for validity on save to decide which links should be dofollow.

    Probably not as bad is also consideration for forums with private content. Some tags could be only visible to logged in members, the replies could be hidden from guests, etc. Having dofollow links to those pages from publicly available pages could also result in negative SEO from search engines constantly hitting protected pages.

    Because of those last 2 aspects, I'm not sure internal links should have dofollow by default. This might be best as a setting, so forum owners have better control on the impact it will have on their SEO.

    I'm sure those points were already raised in a previous discussion.

    With a sitemap enabled, having dofollow or nofollow on user generated content shouldn't matter too much, or does it? I have never tested or seen any report on the best practice from that point of view.

    One other thing I remember from a previous discussion on the subject was the question of external known hosts. Someone willing to enable dofollow on links might likely want to enable it not only on forum links, but also links to their main non-forum website or other websites they manage. This quickly gets very complex for something in core. This part would be best handled by an extension, and as such the whole dofollow feature could belong in that same extension. The only thing core should do is make sure extensions can remove the nofollow rel tag (I'm not entirely sure extensions can do it at the moment because it gets added very late in the rendering lifecycle).

        clarkwinkelmann it's a recommended best practice to apply nofollow on user generated content,

        I would like to qoute this, we should always mark our own domain links as dofollow otherwise bot will just ignore visiting those linked posts/pages. Go through above moz discussion to know about it

        If discussion has 10 replies flarum is telling not to crwal those posts by mentioning nofollow tag

        clarkwinkelmann Bad actors could spam unique 404 links

        No issue, we can keep an a eye on Google search console to make sure there is no issue. If there are 404 links we can redirect them back to correct discussion or home page

        It's not just me who reported this https://discuss.flarum.org/?q=dofollow Since 2015 flarum has this and few others also reported it in 2018

        I have lost hope on flarum SEO i don't think it will be improved, one needs to put lot of efforts to find issues and need to make many requests to get issues fixed?

        Whith your explanation I understand why flarum is doing like this but wordpress is handling the same in a different manner.

        Flarum is such ecosystem which is developer convenience not Google bot friendly, not SEO friendly.

        For all my future needs i will not be choosing flarum.

        I will not start any new discussions mentioning about SEO issues. 🙏

              Hari I see you are very obsessed with SEO but I think you are missing the point here, because you are getting frustrated with things that don't even make much sense, we are talking about internal links and from what I have seen on popular sites this technique is not used either... is it even necessary?

              If discussion has 10 replies flarum is telling not to crwal those posts by mentioning nofollow tag

              This is not true and you probably already know, why any bot would crawl an internal page through a post when they can do it directly from the source? it is not optimal and rarely happen even less with a sitemap.xml, you can do it no doubt, but I think it is obvious that the internal links have already been crawled, because otherwise I don't even know how the user has reached your page.

              Honestly I don't understand the obsession of some people with SEO I mean, the relevance of the content and the speed of the site is the most important thing to be well positioned, that your forum does not appear next to the results of Reddit or Quora is not a question of technical SEO, I can tell you the example of https://forum.webdeveloper.com/, I've run into that Flarum forum in completely random searches on Google lots of times... how do you explain that then? although all the technical SEO is fine the complexity of the search engine algorithms are not that simple to get their attention and the bots are not that dumb either, there are lots of factors mostly non-public for each engine, that's why also the searches vary so much between search engines.

              When Flarum's SEO issues are fixed you will probably be surprised how little effect they have on your site's ranking, but you will see that in the future, obviously it will be addressed at some point, but to me it doesn't seem so critical, in fact any improvement in Flarum's performance would have better effects on SEO than any rel attribute.

                  Darkle I see you are very obsessed with SEO

                  Darkle I don't understand the obsession of some people with SEO

                  askvortsov I find it challenging to understand the obsession with putting things into core that don't need to be there.

                  asking to improve SEO is an obsession?

                  It's okay hari you are not alone https://www.google.com/search?q=developer+vs+seo+meme

                  i request the moderator to close lock this discussion. i no longer want to participate in this discussion.

                          Hari Wow I see you certainly paid attention to what I said in the post, I really didn't mean the obsession in a bad way, it's because you are without a doubt the most SEO concerned person I've seen around here it wasn't a bad thing, but ok whatever, good approach to discuss.

                          • mys likes this.

                            Hari I have lost hope on flarum SEO i don't think it will be improved

                            It will eventually, Better SEO has always been on the roadmap under continuous efforts, it just can't happen any faster than we'd all like at the moment, unfortunately. Someone at some point will sit through it, do research and make an actionable plan for it, as is with all the other roadmap items.

                            Also, the SEO-related requests will help when the time comes.

                            🤣 these memes...

                                  SychO 🤣 these memes...

                                  ROFL they are so good aren't they?

                                  SychO do research and make an actionable plan for it, as is with all the other roadmap items.

                                  So generous of @luceos . he is making an extension to solve the issue, the extension will be out in a few weeks. https://github.com/luceos/flarum-ext-dofollow

                                  We should mark this discussion as solved

                                  And also you should publish Discussion Canonical URL extension here in the discuss whoever needs it will use it. add a strong warning in README file saying "use this only if you are really aware of what you are doing" 😃

                                        A moderational remark: It is important to respect that there are different priorities for different people / communities. The proposal Tag is supposed to support exactly that. Everybody has the possibility to propose what's most important for him / herself. Votes by the community will influence the attention the topic will get from Flarum, or extension developers. Discussion about the content of proposals is greatly appreciated - but should remain focused on the content, which you can also find manifested in our community guidelines:

                                        Ask yourself if your reply adds to the discussion. If it doesn't, give it some more thought before posting.

                                        Hari can you explain shortly what the described issue is and how it affects the Google search results, also why the SEO extension is bad? I've installed the SEO and thought that was enough for Google. Do you mean that with the current implementation of Flarum and the SEO, only the first post in a discussion is indexed and Google won't crawl/index the replies? If so, then I'm with you.

                                            CyberGene only the first post in a discussion is indexed and Google won't crawl/index the replies?

                                            No, all posts are indexed in practice. However there's currently another problem with posts pagination that makes Google search results a total mess unfortunately... flarum/framework3370

                                            I haven't personally experienced issues with discussions not being indexed at all, though.

                                                8 days later

                                                Let me repeat my question from the other discussion:

                                                Correct me if I'm wrong but using links in Flarum posts would potentially affect the ranking of the websites that you link to, not your own Flarum forum. Then why do you care? Furthermore, the question is more specifically regarding internal links, which I understand as linking to a Flarum post in another post on the same Flarum forum. Why would the search engine use that for SEO? It's a self-link and it shouldn't count for the ranking of the same page, otherwise people will start posting such links to improve their SEO.

                                                • Hari replied to this.

                                                  CyberGene i have answered this earlier but i will try to give a another example.

                                                  Assume Google bot is a banker and based on the site content Google gives you more loan (aka a better ranking in Google search results)

                                                  When a banker visits your page for inspection your house (page/discussion) should look good so the bank can consider your page has great value. When the banker finds 3rd part links (links to other websites) will consider as they are your friends. We should always declare 3rd party links as noFollow.

                                                  Coming to other posts links we should display as do follow so the banker visits more rooms (posts) and assumes as our house is so big and we have lot of content. If we have more content it creates scope for spider to consider to give top position in search.

                                                  If your source code has all links including your own sites links as noFollow spider will just ignore crawling.

                                                  No follow > no crawling > no better ranking in Google search > no ranking leads to less web traffic.

                                                  🙂

                                                      Hari I understand all that, I’m a software developer myself.

                                                      I just don’t understand why the crawler needs internal links. The crawler already has all the discussions from the sitemap and will crawl through all the posts in each discussion because the posts are part of the discussion, so it can directly crawl through them. It doesn’t need internal cross-post links to function properly.

                                                          CyberGene crawl through all the posts in each discussion because the posts are part of the discussion, so it can directly crawl through it.

                                                          No, each posts reply hyper link is set to noFollow by flarum that needs to be dofollow. Other wise spider will just ignore going through discussion since it is mentioned as noFollow

                                                          When we scroll down the URL will change like 1, 2, 20th post..etc which is technically a completely different page. Though flarum considers as post link according to SEO it is not.

                                                              Hari there seems to be a contradiction between what you say and what matteocontrini said. Seems like Google is able to crawl through all the posts in a discussion. If that’s the case, then there’s no problem.

                                                                  What @matteocontrini says is there’s some Flarum bug and Google mixes posts but is still able to go through all of them. Which means it can do so without dofollow.

                                                                  • Hari replied to this.

                                                                    Hari I think there are two separate issues here. One is regarding the search bots crawling discussion pages (should be fixed). The other is whether internal links should be dofollow. I’m still not convinced about the latter but I don’t care, it won’t hurt, so whatever Flarum devs decide.

                                                                    • Hari replied to this.