Missing "prev" and "next page" links for crawlers.

Pawel

I am curious, why there are "prev" and "next page" navigation buttons in templates of discussion and index views (flarum/core@v0.1.0-beta.11.1):

flarum/core/views/frontend/content/index.blade.php:19:

    @if (isset($apiDocument->links->prev))
        <a href="{{ $url->to('forum')->route('index') }}?page={{ $page - 1 }}">&laquo; {{ $translator->trans('core.views.index.previous_page_button') }}</a>
    @endif

    @if (isset($apiDocument->links->next))
        <a href="{{ $url->to('forum')->route('index') }}?page={{ $page + 1 }}">{{ $translator->trans('core.views.index.next_page_button') }} &raquo;</a>
    @endif

flarum/core/views/frontend/content/discussion.blade.php:19:

    @if (isset($apiDocument->links->prev))
        <a href="{{ $url(['page' => $page - 1]) }}">&laquo; {{ $translator->trans('core.views.discussion.previous_page_button') }}</a>
    @endif

    @if (isset($apiDocument->links->next))
        <a href="{{ $url(['page' => $page + 1]) }}">{{ $translator->trans('core.views.discussion.next_page_button') }} &raquo;</a>
    @endif

but there is no trace of them in view classes? Were they removed from there or never existed?

I am asking, because I am performing a (brief) analysis of Flarum as it seems to be a good replacement for our community old board and I am trying to tick all the boxes to be able call this migration at the end of a day a successful step in the right direction 😉. I am convinced that those links (which would show up only in the <noscript> node) are very important for content indexing crawlers... Or am I wrong?

Regards,
Pawel

clarkwinkelmann

Pawel in "no-js" mode, pagination is done though ?page= parameters.

In Single Page Application mode (javascript), pagination is done through infinite scrolling.

As to how page links work in Single Page Application mode, for discussions, the link to a specific post also acts as the pagination link and updates as you scroll. For the discussion list there's currently no way of linking to a specific page in Single Page mode, but a PR was submitted to add it flarum/core1829 It presented some challenges though.

Pawel

Thank you for your reply. I am worried, that – probably because of a language barrier – I did not explained my concerns clearly. So please, give me an another try:

Assuming that indexing crawler is not using a JavaScript, it seems for me that without next and prev links to follow, there is no way for it to index posts and discussions beyond the first page... BTW, from what I read, as some people conducted tests of how Google is indexing SPA/dynamic pages, we have got only about 50% of chances that Google bot will trigger generation and/or index content generated by a JS - it sounds legit as I guess it may depend on limitations how much time in a particular moment the bot has for "scanning" the website, how fast the page is responding and so on.

Nevertheless, I suppose that bot cannot scroll the document, so it will not index lazy paginated content. I also suppose that adding links (in the "no-js" content section) to the next pages of paginated content will solve this problem. Following this idea I wanted to add those links in my implementation of Flarum, but it seems that someone thought about that already as I found those links in templates mentioned above. But strangely I did not spot any other code which may fill appropriate parameters in those templates with data which will cause those elements to render... Feeling that solving that mystery is beyond my imagination I asked if someone involved in this project longer than me 😉 could explain that?

Regarding shape of those URL-s, on every paginated page I noticed there is a canonical URL containing the "near" parameter. I thought it may be used as it does not generate repeated content (EDIT: after a research it comes out not to be exactly true and I think that has to be revised).

You wrote that in "no-js" mode there is a pagination (with "page" parameters), but I cannot find any trace of it - on my freshly installed instance of Flarum, but also here - on the official one. Could you direct me to those links (in static document code)?

Taking opportunity - thank you for your extensions and podcasts. 🙂

tankerkiller125

Pawel The templates you referenced are in fact generating the HTML in the no-js version to the appropriate paginated version of the discussion list for the no-js bots and humans with JS disabled. The information for those templates are automatically filled in using the blade templating engine during the response process. You can view these no-script paginations by disabling JS on your forum. If you don't see the pagination links it is because you do not have enough discussions on your forum yet to cause pagination to occur. I believe that pagination happens after 15 discussions/comments.

Pawel

@tankerkiller125 Exactly this is what I am doing. I am disabling JS, opening for example this page: https://discuss.flarum.org/d/20684-flarum-extensions-turkish-language-pack and in noscript#flarum-content element I am getting discussion title and 18 posts (I suppose the count does not equals 20 as 2 comments were deleted?) and nothing more:

<noscript id="flarum-content">
    <div class="container">
        <h2>Flarum Extensions Turkish Language Pack</h2>
        <div>
            <div>
                [...] (here goes the first comment/post)
            </div>
            <hr>
        
            [...]
        
            <div>
                [...] (here goes the 18th comment/post)
            </div>
            <hr>
        </div>
    </div>
</noscript>

There is not any "prev" or "next" button (but the "next" was expected).

When I temporarily re-enable JS to obtain (by scrolling) those URL-s:

https://discuss.flarum.org/d/20684-flarum-extensions-turkish-language-pack/79 or
https://discuss.flarum.org/d/20684-flarum-extensions-turkish-language-pack?near=79

and open them in again JS disabled browser, I will also get that static content with another 20 comments and no prev/next links in noscript block... Maybe I am looking for them in a wrong place?

...

On the other hand, when playing Watson, I see there is a conditional statement in noscript block template:

@if (isset($apiDocument->links->prev))

but I did not find any code which could set this (links->prev) property... (?)

clarkwinkelmann

Pawel you're correct, something seems wrong. I can't see the pagination links in the nojs code for discussions either.

The apiDocument is an API response from an internal call to ShowDiscussionController in https://github.com/flarum/core/blob/master/src/Forum/Content/Discussion.php

ListControllers usually have pagination links added from the data they return. But a ShowController doesn't have pagination links so that's why nothing shows up. I don't remember this controller ever returning links.

If no links were ever returned, then this commit from 2 years ago might be the culprit flarum/core3480a65

Someone should have a quick look if a GitHub issue exists for this, and open one if necessary. I can do it tomorrow.

UPDATE: I tested locally, and it seems to confirm my prediction from above. The last time that link worked was most likely beta 7. I tested on my beta 7 install and links are present. My beta 8 forum is broken and I can't test, but on my beta 9 they are gone. So it's really been that long.