• Extensions

  • FriendsOfFlarum upload, the intelligent file attachment extension

luceos Getting an error after I updated and ran it with map command.

Column not found: 1054 Unknown column 'posts.id' in 'field list' (SQL: select `flbr_fof_upload_fil
  es`.*, group_concat(distinct posts.id) as matched_post_ids from `flbr_fof_upload_files` left join `flbr_posts` on `
  flbr_fof_upload_files`.`actor_id` = `flbr_posts`.`user_id` and `flbr_posts`.`content` like CONCAT('%', fof_upload_f
  iles.url, '%') group by `flbr_fof_upload_files`.`id` order by `flbr_fof_upload_files`.`id` asc limit 1000 offset 0)

    cmsoc i will have to test this with a prefix on the tables

      5 days later

      Darkle php flarum fof:upload --cleanup

      The command wants to delete the files attached to the post while they are attached to the post.

          luceos

          When i add --force it deletes all files already uploaded to posts.

              uwnyuD that should not happen. What is the output of php flarum info.

                  luceos that should not happen.

                  yes, this is happening. All files will be deleted immediately, even though they are attached to a post.

                  luceos What is the output of php flarum info.

                        uwnyuD did you run the mapping first? If you answer [no] to the command, I think you need to run that command separately, or add the flag to run it at the same time. I don't know the flag name though, it's probably described in the README or higher in the discussion.

                            I'm also confused in regards to the mapping option. I was under the impression that performing a --map once would create the required mapping in the database for existing files and you won't have to execute it ever again since the extension will take care to map all new files/posts. However it seems that's not true and you always have to execute the --map option together with --cleanup, right? Or, in other words, there's no mapping between files and posts that is automatically performed and stored in the database with the latest version of the extension. It's only a batch execution that performs a complete mapping every time, right?

                            clarkwinkelmann did you run the mapping first?

                            php flarum fof:upload fof:upload --map?
                            75 matches have been modified between files and posts.

                            What exactly does that mean?

                            I don't think I was able to attach it to the posts.

                            php flarum fof:upload --cleanup
                            it asks each file one by one if you want to delete it. Although this --force delete all attachments but also what you should keep.

                                luceos use both --cleanup --map at the same time..

                                Maybe I misunderstood. But I still don't understand. Doesn't it automatically clean them all by itself?
                                Is there an option to automatically clean it every hour?

                                After --cleanup it asks if I should do joint mapping instead. If I choose yes, it adds --map to the original command and asks to delete the file along with it. If I choose no option, the original command remains, without mapping, but the files still have to be deleted one by one by typing yes, but the files requested to be deleted are attached to the post, so I don't understand why they ask. If I add --force it clears everything. I was thinking of deleting the files automatically so that what is not attached to a post is deleted. But he constantly thinks he is not attached to anything.

                                    uwnyuD use all three flags to auto delete and first map files to posts: --map --cleanup --force or isn't that working?

                                        luceos Its now deleted all files and database tables in fof_upload_files. Even what was attached to the post. (I have backup.)

                                          In fof_upload_file_posts mysql table should there be something? It's empty for me.

                                          @luceos preparing for the 1.7 update I'm wondering if it's OK to change in my composer.json from "fof/upload": "^1.3.0-beta.1" to "fof/upload": "*"? Will that also switch me to the official release of the extension once it's ready?

                                          P.S. I set it to * and now after the upgrade to Flarum 1.7.0 the fof-upload extension got downgraded to 1.2.3. I hope composer knows what to do and there won't be any problems with the downgrade. Or not?

                                            CyberGene most extensions are not designed to be downgraded so I would not recommend downgrading the extension once you installed and use the 1.3 beta of FoF Upload.

                                            There shouldn't be any issue keeping the beta constraint on FoF Upload, it won't prevent you from updating Flarum. The instruction regarding * in the Flarum update instructions is a general best practice to avoid Composer update errors before they happen but not strictly required.

                                            You should be able to use *@beta as the constraint for an extension if you want to always allow installation of the latest version, even if it's a beta.

                                              14 days later

                                              Is it possible to verify the uploaded files? If a single file is uploaded multiple times, it will cause the same file to be stored repeatedly and take up too much space.

                                              For example, here is a file foo.jpg, when they are uploaded multiple times, they will become urls like https://domain/uploadDate/timestamp-fileId-foo.jpg , due to changes in timestamp and fileId, files with the same content will be uploaded multiple times.

                                              It might sound like a small problem. But when the file is uploaded too many times, or the size of the uploaded file is too large, things are not so simple. Therefore, I think it is necessary to increase the verification of the file (such as hash, md5, sha1, etc.).

                                                Burial0268 is it happening often on your forum?

                                                When the same user wants to reuse a file, they can pick the file from their gallery, in which case it won't be re-uploaded.

                                                If many users are uploading the exact same large file, what is the use case? If users are exchanging the same ressources again and again maybe a different extension should be used with a central download section where every user can contribute files instead of blindly uploading the same thing without checking what has already been uploaded. Or just generally enforced through rules and moderation.

                                                De-duplicating files across different users would add a lot of complexity (visibility and access control) to the extension for little gain. If this really needs to be done in FoF Upload this would be best solved by storing files on a filesystem that transparently de-duplicates files so that the extension doesn't need to know about it at all.