@luceos can you explain in more details how the new functionality works and probably update the first post in this discussion? Currently the commands are in a post somewhere in this discussion and it's hard for me to find it among 2000+ posts (BTW, here's where load while scrolling for Flarum has its downsides, it's just too awkward and for some reasons while loading the browser skips some posts, so it's not even smooth).

In particular can you explain what the --map option does? My understanding is you should execute it only once after you upgrade to the new dev in order to map the existing files and posts (and the mapping will be stored in the database for future use), thus there will be no need for second execution of the --map, is that correct? If not, does that mean that I should always combine --map with --cleanup? If I execute only --cleanup with no --map, what will be the result? Are the files physically deleted from the corresponding storage, e.g. AWS? I remember there were some errors with aws-s3 vs aws_s3 in the DB or something like that and you said you're going to implement a proper migration. Is that ready now? Can I safely execute the cleanup logic on my prod forum or is that functionality still in beta?

Again, a better description of the algorithm and the commands would be very helpful since I am confused and still worried to use it. I haven't done so anyway, because of the DB discrepancies and the errors, as well as fear that the functionality is still in beta.

P.S. To add to the above, executing --cleanup without --map results in the following warning:
Are you sure you want to clean up without mapping first? (yes/no) [no]:

So, what's the difference, what will happen in the two scenarios, with and without mapping? Am I gonna lose posts/files?

P.P.S. So, I still get the In Manager.php line 85: No adapter configured for aws_s3. Is it safe to update my DB records? But I think this should have been a migration since I have been using only official releases of extensions. And besides, all my new uploads keep on using the aws_s3. What extension/code is inserting records in fof_upload_files table with upload_method set as aws_s3 and why is the extension expecting it to be aws-s3 instead? Not sure that's OK.

CyberGene Is the mapping functionality capable of detecting orphaned files from before the new development was introduced? By orphan files I mean those files that are not used in any post and whose media library entities have been deleted by the user? In my understanding what remains is just a file on the storage that nobody knows about?

If no entry is available in the database, then yes, Upload will not know about it and cannot delete those files sadly.

CyberGene If the answer to the above questions is no, then is there a way to detect such files? I guess it's a bit tricky since such a functionality would have to list all the files on the storage and check whether there are links to them in any post.

Possibly, if a dedicated S3 bucket was used we could create an index of the files that are known and go over all files in the S3 bucket to delete that are unknown. It will be a painful process as this will certainly hit api limitations on large buckets/files.

CyberGene With the latest version of the extension, if someone uploads a file and never uses it in a post and then deletes the media entity, what happens? I would assume that in such a case the file is still mapped correctly, so that it can be cleaned up without the need to explicitly perform a new mapping, right?

Yes it will be deleted if you use the command.

Please understand that this is a first step into the direction I wished I had taken at the inception of Upload, namely to keep track and remove orphaned files. If at any point these files remain, we hope to find a solution to tackle those use cases too.

CyberGene I encountered again the error No adapter configured for aws_s3. So, the migration fixed the existing entities, however the newer ones keep on being stored with the aws_s3 adapter. Can I perform the migration again to fix the newer entities or a migration is a one-off thing that won't happen again unless I upgrade an extension?

If you remove the db entry with the same name from the migrations table, you can. I would love to hear whether it works the second time, because it should not happen.

delete from migrations where migration = '2023_08_20_000000_alter_uploads_s3_adapter.php';

Darkle By the way, I don't remember it very well (I think it was discussed at some point), but if I remember correctly this system maps the files internally, so now that it's stable it will eliminate the need to show the id in the filename?

Yes it stores the url in the table, so it tries to find that in posts.

Darkle Upgraded and did another cleanup with an R2 bucket, all good! Although I can confirm what CyberGene mentions, the migration updated the previous files with aws_s3 to aws-s3, but the newly uploaded files continue to add aws_s3.

Aha I see, I will look into it.

Check all uploaded files

Is there a way to see all uploaded files (from all users) in one go?

As an admin I would like to be able to see what was uploaded on my server and delete it fast.

How can a user delete files?

If I try to delete a file as a user in my library, a dialog says it will only be hidden. Why? Even if I try to delete something, what wasn't used in a post, it will be only hidden.

This is fatal! Because I really don't want to have ghost/garbage data on my server.

Is compression possible

The crop avatar extension has a builtin compression. It would be nice to have something like this for the images, which gets uploaded, too.

Delete unused files frequently

Is it possible to delete files, which aren't used in the forum with a cronjob?

Illegal usage of file uploads

The possibility to use the upload can be divert from its intended use. People could upload illegal/copyrighted/pornographic/pedophile media without the knowledge of the admin and use it somewhere else to share it.

Another bad thing is, that a new user could upload stuff, even if first approval is active. For uploads there is no approval needed and can be done while composing a post.

Is there a way to control or check this? It would be already helpful if the things work, which were mentioned above (e.g. delete unused files automatically).

Thank you