Decontaminator by Flarumite

[deleted]

IanM Just circling back here to let you know that all is working as expected. Performed various tests this morning and all work.

I have a suggestion though. Is it possible to add a class (style) to the replacement text ? For example, something like "[redacted for privacy reasons]" should stand out a bit more - for example

[redacted for privacy reasons]
[redacted for privacy reasons]
[redacted for privacy reasons]

With a CSS style, it'd be possible to much more with the text I think which would be an effective enhacement.

IanM

[deleted] Just circling back here to let you know that all is working as expected. Performed various tests this morning and all work.

Excellent, I'm glad it's working out for you now 🙂

[deleted] I have a suggestion though. Is it possible to add a class (style) to the replacement text ?

That's a fantastic idea. I've created an issue, and I'll start looking into it flarumite/flarum-decontaminator3

IanM

[deleted] I have a suggestion though. Is it possible to add a class (style) to the replacement text ?

I've been playing around with this idea a little, and would value your opinion on this, please:

Replaced content is wrapped in a span, with a class assigned to it. Also added a tooltip with translatable string.

Is this along the lines of what you had in mind?

iPurpl3x

Hi @IanM,
I have tried to setup this extension but my regexp do not work... They work fine on https://regex101.com/.
I have tried the one you provided for e-mails /\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}\b/mi and it worked.

I have made slight change to your email regex to match more cases and it doesn't do anything:
\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]+\b/mig
And I have this other one, meant to hide phone numbers and it does not work either:
(\b(([0-9]{4})|0|1)|\B\+[1-9]{1,3})(\s?\(0\))?(\s)?[1-9]{2}(\s)?[0-9]{3}(\s)?[0-9]{2}(\s)?[0-9]{2}\b/mig

Last thing: when I tested the one that worked, I also activated the Flag option. This gave me a Permission Denied 403 error when trying to make a post. Once I disabled the Flag option it worked fine.

IanM

iPurpl3x Thanks for checking the extension out 🙂

iPurpl3x I have made slight change to your email regex to match more cases and it doesn't do anything:
\b[a-z0-9._%+-]+@[a-z0-9.-]+.[a-z]+\b/mig

iPurpl3x And I have this other one, meant to hide phone numbers and it does not work either:
(\b(([0-9]{4})|0|1)|\B+[1-9]{1,3})(\s?(0))?(\s)?[1-9]{2}(\s)?[0-9]{3}(\s)?[0-9]{2}(\s)?[0-9]{2}\b/mig

Couple of things here. Firstly, you need to make sure that the regex begins with /, and also you don't need the g for global either. By modifying your examples to the examples below, I was able to test them successfully.

/\b[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]+\b/mi
/(\b(([0-9]{4})|0|1)|\B\+[1-9]{1,3})(\s?\(0\))?(\s)?[1-9]{2}(\s)?[0-9]{3}(\s)?[0-9]{2}(\s)?[0-9]{2}\b/mi

iPurpl3x Last thing: when I tested the one that worked, I also activated the Flag option. This gave me a Permission Denied 403 error when trying to make a post. Once I disabled the Flag option it worked fine.

This is interesting, potentially the user who posted the matched content does not have flag permissions, would that be correct? I might need to revisit how the flag is raised in this case

iPurpl3x

IanM Couple of things here. Firstly, you need to make sure that the regex begins with /, and also you don't need the g for global either. By modifying your examples to the examples below, I was able to test them successfully.
/\b[a-z0-9._%+-]+@[a-z0-9.-]+.[a-z]+\b/mi
/(\b(([0-9]{4})|0|1)|\B+[1-9]{1,3})(\s?(0))?(\s)?[1-9]{2}(\s)?[0-9]{3}(\s)?[0-9]{2}(\s)?[0-9]{2}\b/mi

Great thank you very much for noticing this ! It works now. I guess I was just blindly copying the regex from https://regex101.com/ and it came without the /. My bad.

IanM This is interesting, potentially the user who posted the matched content does not have flag permissions, would that be correct? I might need to revisit how the flag is raised in this case

This is right, in my configuration, users are not allowed to flag their own posts.

IanM

iPurpl3x

iPurpl3x Great thank you very much for noticing this ! It works now.

No problem!

iPurpl3x This is right, in my configuration, users are not allowed to flag their own posts.

Ok, I will make a change to how this scenario is handled. Won’t take too long...

IanM

0.1.3

Better handles cases where users do not have permission to flag their post when matched content is found (iPurpl3x)

Updating

composer require flarumite/flarum-decontaminator
php flarum cache:clear

iPurpl3x

IanM Better handles cases where users do not have permission to flag their post when matched content is found

In my case it still doesn't work unless I allow users to flag their own posts in the Flags extension settings.

IanM

iPurpl3x I'll take another look at this today

IanM

0.1.4

iPurpl3x this should now fix your issue

Updating

composer require flarumite/flarum-decontaminator
php flarum cache:clear

iPurpl3x

IanM this should now fix your issue

Yes this time it works! Thank you for your quick bugfixing 🙏

IanM

iPurpl3x You're welcome 😃

Kyrne

[deleted] Ian is welcome to take anything he wants from FoF/filter. This ext honestly looks better than mine. If he does I'll deprecate mine in favor of his.

IanM it would be cool to have the ability for admins to allow users to decide if they want things to be censored. New users should totally have a SFW experience by default but many may not mind seeing the filtered words.

[deleted]

IanM yes, and better. The span class is more than enough for styling and the tooltip is the icing on the cake. Will the tooltip text be customizable ?

iPurpl3x

[deleted], IanM In my opinion, the translatable string should be the one that is displayed in the text (as <censored>) and then the text of the tooltip should be the one that says what kind of data has been censored (set by the admin).
Now one thing will still bother me: for multi-language forums, how can you have translated strings for this second part that says what has been censored? You would need multiple fields, one for each language for the admin to configure it right...

[deleted]

Not sure if I should use fof filter or yours, I like some features of filter and yours. Can't you guys fuse? 😃

Mark73

[deleted] Same here, I'm enjoying the fof filter extension as it has one feature that I can't leave behind, which is auto-merging double posts. @Kyrne has done some great work. However, the filtering system of fof's extension has some issues, appropriate words containing bad words in them get flagged, e.g. glasses has a bad word in he middle so it gets flagged. Thus why I'm currently using it for auto-merging only. I still think the two extensions are slightly different as fof's extension only censor strings that you type in yourself but this one seems to be more advanced.

IanM

iPurpl3x is shouldn't be too hard to add special cases to cover this. I'm a little snowed under for the next week or so, but will take a look as soon as I can. Alternatively, feel free to take a look and open a PR, if you have time 🙂

[deleted] Yes, the tooltip text is provided as a translation string, so can be customized to your liking. I'm still a little undecided on the tooltip however, so I'll likely only include the span in the next release, whilst I continue to think it over.

[deleted] What i also like a lot is the email notification, not sure if decontaminator has that

No, decontaminator does not have email notifications. It did in an early pre-release version, but I replaced that with the flarum/flags integration instead.

iPurpl3x Now one thing will still bother me: for multi-language forums, how can you have translated strings for this second part that says what has been censored? You would need multiple fields, one for each language for the admin to configure it right...

Indeed, I'm very open to ideas and suggestions on how multi language forums could be handled.

Kyrne Ian is welcome to take anything he wants from FoF/filter. This ext honestly looks better than mine.

Awww thanks 😊

[deleted] What do you think about the above posts?

As always, I'm willing to consider any reasonable suggestion. I guess it should be pretty easy to add the double post merging into decontaminator. Again, feel free to PR, or I will take a look when I can.

I'm in 2 minds about a simple/advanced mode, as the primary goal of this extension is to reliably filter content such as PII and sensitive data at source. This is one reason why the extension does not record the data it matches. I'm sure @[deleted] could attest to the need for this too.

[deleted]

Honestly, I love the auto merging and the "simple" mode with only using words. Regex can sometimes be a pain 😃
What i also like a lot is the email notification, not sure if decontaminator has that 😃

Mark73

[deleted] Oh yes the email notification is surely a great feature that I forgot to mention. But still, would you like to find out that tens of appropriate posts are waiting approval every time you check up your forums? I myself wouldn't

[deleted]

Mark73 Yeah well, maybe a simple mode and a regex mode would be nice, so you can decide. Sometimes admins want to be extra safe i guess 😃

IanM

0.1.5 0.1.6

Add a <span> element around replaced content ([deleted] )
Add a tooltip on replaced content (translatable)
Add filtering to user bio Requires fof/user-bio 0.2.0 or later

Note: Only newly replaced content will be marked with the span and tooltip

Updating

composer require flarumite/flarum-decontaminator:^0.1.6
php flarum cache:clear

Edit: Apologies, forgot to build the JS 🤦‍♂️

« Previous Page Next Page »