There should be a CLI command, and maybe eventually a UI, to repeatedly toggle half the remaining extensions and ask the user whether a problem is there. This would massively reduce the time needed to find problematic extensions on communities with more than a few extensions, which is probably the vast majority of Flarum communities.

The "search space" for bisect binary search could just be the dependency-resolved order.
At each iteration, after setting the proper subset of extensions to be enabled/disabled, we should:

  • Recompile/republish assets
  • Clear cache
  • Maybe something else

Before starting this process, we should write the currently enabled extensions to a file, so that the forum can recover if the bisect is aborted. This file (e.g. bisect_state.json) could then be extended to power a web UI version of bisect, although we'd want to keep the CLI one around too.

If a community has 10 extensions, finding the culprit would take 3-4 steps
If it has 30, finding the culprit would take 4-5 steps.
At 100, it would be 6-7, and at 300, it would still only take 9 steps. Yay base-2 log!

A slightly more sophisticated version could take one (or several) extensions that MUST be kept enabled and should not be considered in the binary search; this could be used to find which problematic extension is causing issues when interacting with a working extension.

I was thinking about this recently.

My issue is that I have no idea how to write the binary tree to take into account extension dependencies between each other. Is there even a guarantee that's it's always going to be possible? If you start disabling extensions without verifying the dependencies it's almost certainly going to lead to a 500 error that's unrelated to the original problem being investigated.

    clarkwinkelmann Dependency resolution performs topological sort; by definition, if ext b follows ext a, then ext a cannot depend on ext b. So if we perform binary search on the list of dependencies, that shouldn't be an issue.

    To implement the "extensions must be kept enabled" version, we could:

    • Take the top-sorted list of extensions
    • Recursively find all dependencies of the mandatory extensions, and extract them into a second list

    The second list would be kept enabled throughout the binary search on the first list.

      This is a very good idea and will help a lot in supporting all users.

      I'm glad this was put in writing, I had a similar idea a while ago although I think the dependency logic would indeed make things slightly more complicated. I'd also would like to add that it is easy for this tool to check the http status code of the home page or attempt booting the InstalledApp logic manually to check against Exceptions, this would already help prevent issues where browser/webserver/proxy cache play a role..