Flarum Query: extensions big data analysis

clarkwinkelmann

I'm back with another experimental service for Flarum.

This tool lets you query through a large amount of data on all Flarum extensions.

https://query.flarum.dev/

What is the purpose

Get cool statistics about the Flarum ecosystem.
Find which extensions depend on each other or translate each other.
Find extensions that use deprecated code.
And pretty much anything you want.

Things to know

Database is MySQL. Use any valid SQL. The user has SELECT permissions on the database only.
For performance reasons results are limited to 100 rows under most circumstances.
Data is updated twice per day.
Data is pulled from Packagist. ZIPs are downloaded from GitHub for each release. For now only packages whose dist files are available on GitHub are parsed for details.
Source usage, javascript imports and PHP imports are experimental. Not all imports are detected at the moment.
Each table has a column for package and version. The latest version of each extension is duplicated with a second copy using version = latest. Most queries will probably use a where version = 'latest' clause unless you want to query historical data.

Guidelines

Don't run queries that are likely to impact the server performance.
Don't use automated tools to access the data.
I reserve the right to delete any saved query with inappropriate title or comments
Please reach out to me privately if you find security issues.

All requests performed are logged.

It's all about sharing great requests ! If you find interesting data, please post a link below !

Queries saved with "public" option might show up on the homepage in the future. Queries saved without "public" can only be seen by people with the link.

I might add user profiles in the future so you can manage your saved queries.

Examples

Translations in unexpected namespaces
https://query.flarum.dev/q/33bf8191-b09e-411c-9f58-703b5aec7935

Routes defined by multiple extensions
https://query.flarum.dev/q/4a2e849b-9e68-47e4-b394-5a739ef70e24

Extensions not allowing beta 12
https://query.flarum.dev/q/76e44bbc-0c0d-4c91-b600-fab8ef57bb6e

FoF packages using Zend classes (must be migrated to Laminas before beta 13!)
https://query.flarum.dev/q/653a7856-dc90-44f1-8462-155cff9402d1

Language packs with most translated extensions
https://query.flarum.dev/q/3ca3c304-9999-44a0-b07d-151c708bf1bf

Possible improvements

I will try to improve the javascript and PHP namespace recognition. I might bring in some library designed for that. It's all regex-based at the moment.

There's currently no way of telling forks aside from normal extensions, so it can mess up some stats about invalid namespaces and such as it's expected that forks will break those rules. Forks are usually not allowed on Packagist, but here we are...

There's also no easy way of telling which extensions are actually maintained or compatible with a given Flarum version. Your best bet at the moment is to compare the requirement constraints or release date to limit to extensions updated recently.

I plan to add a page listing curated requests, and another page listing the most recent public saved queries.

Let me know if you have any suggestions !

clarkwinkelmann

Following this report https://discuss.flarum.org/d/23055-fofupload-extention-error I wanted to check if other extensions are also missing requirements for Guzzle.

This tool comes to the rescue https://query.flarum.dev/q/fe2cee08-ba8b-4bc7-867d-60ae00e3dbd7

There are still some limitations. For example any auth extension that requires a league oauth driver would not need to require Guzzle as it's already required by the league driver. Also if any extension uses Guzzle without using a use statement, it will not be picked up by Query.

I also did not include whether an extension is abandoned in the output of the query.

rob006

clarkwinkelmann For example any auth extension that requires a league oauth driver would not need to require Guzzle as it's already required by the league driver.

That is not true. Dependencies are not part of the BC promise - league driver can remove Guzzle dependency at any time. Extensions should explicitly require all dependencies they're directly using.