Bots hammering my /vendor directory

endlessedits

I am an active and enthusiastic Flarum user but not at the technical level of most of you posting here (developers, etc.).

I have been reviewing my logs and it seems there are concerted efforts to "GET" specific resources in my /vendor directory relating to extensions.

My installation is dutifully returning a 403 error (forbidden request) to them.

Is there anything for me to worry about, is my security at the proper level? Did I do anything wrong to have this happen, or are people just aware of what to try to hit Flarum installs with? What harmful data will they get if they are successful?

Thanks in advance for any responses.

Here is a very small sample of what they are trying to get:

/vendor/the-turk/flarum-pallet-theme/js/src/admin/
/vendor/s9e/text-formatter/src/Configurator/Traits/
/vendor/doctrine/dbal/lib/Doctrine/DBAL/
/vendor/the-turk/flarum-pallet-theme/less/forum/
/vendor/s9e/text-formatter/src/Configurator/Collections/
/vendor/clarkwinkelmann/flarum-ext-author-change/resources/
/vendor/symfony/console/Command/
/vendor/franzl/
/vendor/egulias/email-validator/src/Parser/

luceos

adrianm I'm curious about how /vendor is exposed at all, as the document root for your site should be /public which is on the same level as /vendor?

That is indeed the recommended way to install. But installing Flarum inside public is also possible.

endlessedits Here is a very small sample of what they are trying to get:

What worries me is the fact they seem to know exactly that these folders and files exist. This shouldn't be possible. Can you share a section of the access logs here, because they include the response status code as well.

adrianm

Hi @endlessedits I'm curious about how /vendor is exposed at all, as the document root for your site should be /public which is on the same level as /vendor?

adrianm

Sounds like they might be getting a directory read and then iterating through everything. Total guess though of course. Either way you would want to lock the server down a bit more. On the flarum side maybe a htacess would help to make sure requests to things that should not be accessed are dropped.

luceos

adrianm On the flarum side maybe a htacess would help to make sure requests to things that should not be accessed are dropped.

There's a htaccess that takes care of that, the installation instructions state the following as well:

Simply move all the files inside the public directory (including .htaccess) into the directory you want to serve Flarum from. Then edit .htaccess and uncomment lines 9-15 in order to protect sensitive resources. For Nginx, uncomment lines 8-11 of .nginx.conf.

https://docs.flarum.org/install#customizing-paths

endlessedits

Thanks to both of you for the quick and helpful responses. I have had a suspicion that my knowledge of .htaccess and directory permissions is lacking. I had an extremely difficult install process and it was long enough ago that I don't recall all the trial and error fixes I implemented, but my Flarum installation has been working perfectly for 6 months or more at this point.

I have Flarum operating directly at my-domain-name.com, with the composer.json, composer.lock, config.php etc. all in the public_html directory. The permissions on /public_html are 751.

/assets, /storage and /vendor are all in the same place, under /public_html as well. All of the permissions for these 3 subdirectories are 755. It appears I have uncommented the lines designated when "not using a 'public' directory." .htaccess file permissions are 644.

The first lines of .htaccess are:

<IfModule mod_rewrite.c>
  RewriteEngine on

  # Ensure the Authorization HTTP header is available to PHP
  RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

  # Uncomment the following lines if you are not using a `public` directory
  # to prevent sensitive resources from being exposed.
  # <!-- BEGIN EXPOSED RESOURCES PROTECTION -->
  RewriteRule /\.git / [F,L]
  RewriteRule ^auth\.json$ / [F,L]
  RewriteRule ^composer\.(lock|json)$ / [F,L]
  RewriteRule ^config.php$ / [F,L]
  RewriteRule ^flarum$ / [F,L]
  RewriteRule ^storage/(.*)?$ / [F,L]
  RewriteRule ^vendor/(.*)?$ / [F,L]
  # <!-- END EXPOSED RESOURCES PROTECTION -->

  # Pass requests that don't refer directly to files in the filesystem to index.php
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^ index.php [QSA,L]
</IfModule>

--end snippet of .htaccess file

Here's one of the log entries of a 403 error where googlebot tried to reach within /vendor:

66.249.74.xxx - - [26/May/2024:06:45:59 -0700] "GET /vendor/guzzlehttp/guzzle/src/ClientTrait.php HTTP/1.0" 403 3322 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.6422.65 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

I'd surely appreciate any advice for making changes to the above permissions or .htaccess.

luceos

endlessedits HTTP/1.0" 403 3322

403 is forbidden, so there's no risk at all that these files are accessed from the outside. You can also try running Clarks lab to check for any other security issues, it's at https://lab.migratetoflarum.com/

endlessedits

Thanks. Using Clark's site, I received a report that there were problems with vendor folder and composer files. (Detail showed composer.json serving a 200 and composer.lock a 403 (go figure, as I believe they are subject to the same rewrite rule (see it above)).

I changed the permissions on composer.json and composer.lock from 644 to 640, cleared the cache, and the "composer files" error on Clark's site disappeared. My Flaurm installation continued to work.

The vendor directory warning remained.

So then, next, I changed /vendor directory permissions from 755 to 750, cleared cache, and the Flarum install stopped working (blank white page only). Same with 754. In both cases the vendor directory problem was still reported by Clark's site.

Trying permissions of 751 and 753 for /vendor kept the site working but /vendor still reported as being exposed.

I did NOT select applying the permissions changes recursively, would this help? I am afraid to make changes to so many files that I could not easily reverse to whatever they were before (assuming they might vary).

Is there an obvious fix I could make to this?

RewriteRule ^vendor/(.*)?$ / [F,L]

Thanks in advance. Trying to learn regular expressions to better understand rewrite rules is a big undertaking way outside of my normal day-to-day existence. (Maybe someday . . . )

P.S. Clark's site detail is showing one file within /vendor that is getting a 200 code, and I can access it myself when using the URL for it.

clarkwinkelmann

endlessedits as long as Apache mod_rewrite module is enabled, the provided RewriteRule and regex should cover everything you need.

I took a look at the last private scans on the lab which I assume are for your website. The last scan complained about vendor/composer/installed.json but it seems like it's properly secured now. I see your server has a custom 403 response page, but this shouldn't have any impact on this working correctly.

It would be strange for only some of the vendor files to be protected if the mod_rewrite is properly enabled... If you still have issues, try rebooting the server/webserver if that's something your host allows.

The filesystem permissions on disk shouldn't have any impact on this either. Apache (or PHP, but usually they run under the same shell user) should be allowed to read all those files as they are necessary to run Flarum. The htaccess just tells Apache it's now allowed to give those files to a visitor.

endlessedits

Thanks for checking, yes, that was me. It's a nice tool.

I am the perfect example of knowing just enough to be dangerous. Then I get to learn new things by having to corrrect my unforced errors, at the expense of much time and pain.

endlessedits

OK, it was still driving me crazy, so on an entirely different VPS (using same Linux flavor and version as my Flarum site) I did a clean install using the same control panel (Hestia) -- using the built-in Hestia Quick Install button, accepting all defaults (such as using the / directory for the install (e.g., /public_html).

I ran it against Clark's website and got the same errors as on my primary site, for /vendor directory and composer files, with a D grade.

I am able to see in my browser the content of at least one file within the vendor folder, as before.

I note that the .htaccess file written by the Hestia Quick Install script properly uncommented the recommended rewrite commands.

At this point, at least I think it's not anything I did. Perhaps it is a Hestia issue (their install script), or something else.

Fortunately, yesterday after mangling my install by changing permissions and getting the white screen of death for three hours I was able to test my Hestia restore backup function and it worked like a charm.

My solution to these security concerns (which may or may not be meaningful) will be to make super-frequent offsite backups and only use that VPS for my Flarum installation.

It needs to be said that whatever Hestia has done to make Flarum install so easily was/is appreciated by me. It was the only CP that allowed for a successful import by me of a prior installation elsewhere.

My unrequested two cents on installation/importing in 2.0 would be that a script to import an existing .sql and customization of Flarum (images, extensions etc.) from somewhere and to install afresh and place it in an arbitrary directory (e.g., move an install from a /public directory and optionally be able to create it somewhere else, such as in /public_html, etc.) would be a godsend and increase non-power user confidence in not being locked into a particular VPS/server situation, especially since we don't presently seem to have any export tools to move large Flarum forums to other forum software.

All that said, I love Flarum and am amazed how well it works once installed. Thanks guys!

endlessedits

Getting a "D" on https://lab.migratetoflarum.com has been driving me crazy, and despite all the efforts above, it went unsolved until this weekend.

Using a separate, very underpowered VPS (1gb memory) I clean installed Debian 12, added Hestia CP and on the second try I decided to UNCHECK the Apache install checkbox for the installation script maker. https://hestiacp.com/install.html

As it happens, Apache still ends up being installed, as is Nginx (as reverse proxy). The Hestia "Quick Install App" function installed Flarum and, after doing the initial config, voila, it received an "A+" from https://lab.migratetoflarum.com.

I then was able to install my existing Flarum files and now have a complete backup which gets a higher MigratetoFlarum rating than the original.

My best guess is there is something in Debian 12 or Hestia's Apache install that is different from what you get when you uncheck Apache in the Hestia install script. What is somewhat interesting to me is the .htacess file has the EXPOSED RESOURCES PROTECTION rewrite rules active (un-commented out), but the Nginx.conf file does not remove the pound signs that comment-out those rewrites.

So I would guess something in the Hestia Apache install is interfering with the expected working of the .htacess rewrite rules.

Once I gain some more confidence in restoring backups I will try to reinstall my main VPS and Flarum and will report if it also works as described above.

Bottom line - if you like Debian 12 and the free Hestia CP (I have had great luck with both for Flarum), try UNCHECKING the Apache install option in the Hestia installation script maker at https://hestiacp.com/install.html.

I hope this helps someone. Thanks to all who tried to solve it from the Flarum side, seems it wasn't a Flarum issue after all.