Page 1 of 1

Profanity filter

Posted: January 11th, 2012, 12:40 am
by Jonty800
Swearing on servers can be annoying, so I thought I'd share this.

Its a simple profanity filter that I wrote. It works by replacing any bad words with a word of your choosing.
The filter works by replacing full words, that way people can still say words like "Scunthorpe"

1. Add a permission called "Swear" in the Permissions.cs
2. Go to chat.cs. Add these two at the top of the page.

Code: Select all

using System.Text.RegularExpressions;
using System.IO;
Add just above public static bool SendGlobal:

Code: Select all

public static List<string> Swears = new List<string>();
        public static IEnumerable<Regex> badWordMatchers;
Then you can add this into SendGlobal (Caution: contains swears)

Code: Select all

if (!player.Can(Permission.Swear))
                {
                    if (!File.Exists("SwearWords.txt"))
                    {
                        StringBuilder sb = new StringBuilder();
                        sb.AppendLine("#This txt file should be filled with bad words that you want to be filtered out");
                        sb.AppendLine("#I have included some examples, excuse my language :P");
                        sb.AppendLine("fuck");
                        sb.AppendLine("fucking");
                        sb.AppendLine("fucked");
                        sb.AppendLine("dick");
                        sb.AppendLine("bitch");
                        sb.AppendLine("shit");
                        sb.AppendLine("shitting");
                        sb.AppendLine("shithead");
                        sb.AppendLine("cunt");
                        sb.AppendLine("nigger");
                        sb.AppendLine("wanker");
                        sb.AppendLine("wank");
                        sb.AppendLine("wanking");
                        sb.AppendLine("piss");
                        File.WriteAllText("SwearWords.txt", sb.ToString());
                    }
                    string CensoredText = Color.ReplacePercentCodes("&CBlock&F"); //you can change this. Always end in &F though
                    

                    const string PatternTemplate = @"\b({0})(s?)\b";
                    const RegexOptions Options = RegexOptions.IgnoreCase;

                    if (Swears.Count == 0)
                    {
                        Swears.AddRange(File.ReadAllLines("SwearWords.txt").Where(line => line.StartsWith("#") == false || line.Trim().Equals(String.Empty)));
                    }

                    if (badWordMatchers == null)
                    {
                        badWordMatchers = Swears.
                            Select(x => new Regex(string.Format(PatternTemplate, x), Options));
                    }

                    string output = badWordMatchers.
                       Aggregate(rawMessage, (current, matcher) => matcher.Replace(current, CensoredText));
                    rawMessage = output;
                }
                }
3. You can replace string CensoredText with whatever word you like, so if i swear it will now say "block off you block". Feel free to use color codes too, as long as the string ends with %F or has '+"&F";' at the end.

4. In the configGUI, go to ranks and then you can tick "Swear" to each rank that you would like to be able to swear on the server. (Note the filter only works in general chat, players can still swear in staffchat and pm's.

5. When you start the server, when someone firsts talks, a new file in the root of your server will be made called "SwearWords.txt". When the server is shutdown, you can edit this file adding new swear words that you wish to be filtered. Make sure these are on new lines.
Since the filter works using full words, make sure you add things like f**k and f**king (ignore the asterixis). A list of default words will be added when the file is first made.

Re: Profanity filter

Posted: January 11th, 2012, 1:52 am
by fragmer
This will work, but I'm worried about performance. Current code does disk i/o and may create hundreds of temporary objects for each message. I'd cache the list and especially the Regex objects.

Also keep in mind that the whole chat-censorship effort is completely futile. If you censor "fuck" people will still write "fvck", or "fukk", etc.

Re: Profanity filter

Posted: January 11th, 2012, 2:52 am
by Jonty800
so would loading the profanity list on start-up solve this? So when the server starts, the SwearWords.txt is placed in a cache so the list doesnt need to be loaded again?

edit: nvm I just used a list

Re: Profanity filter

Posted: January 11th, 2012, 3:28 am
by Bloody_Llama
I would use it to replace "pony", "mlp", etc with very bad puns.

Re: Profanity filter

Posted: January 11th, 2012, 3:38 am
by Jonty800
I made a cache for the list and for IEnumerable<Regex> badWordMatchers

I can't see anything else to cache. Does this look ok?

Code: Select all

Top of the page: 
public static List<string> Swears = new List<string>();
        public static IEnumerable<Regex> badWordMatchers;

In SendGlobal:

if (!player.Can(Permission.Swear))
                {
                    if (!File.Exists("SwearWords.txt"))
                    {
                        StringBuilder sb = new StringBuilder();
                        sb.AppendLine("#This txt file should be filled with bad words that you want to be filtered out");
                        sb.AppendLine("#I have included some examples, excuse my language :P");
                        sb.AppendLine("fuck");
                        sb.AppendLine("fucking");
                        sb.AppendLine("fucked");
                        sb.AppendLine("dick");
                        sb.AppendLine("bitch");
                        sb.AppendLine("shit");
                        sb.AppendLine("shitting");
                        sb.AppendLine("shithead");
                        sb.AppendLine("cunt");
                        sb.AppendLine("nigger");
                        sb.AppendLine("wanker");
                        sb.AppendLine("wank");
                        sb.AppendLine("wanking");
                        sb.AppendLine("dick");
                        sb.AppendLine("piss");
                        File.WriteAllText("SwearWords.txt", sb.ToString());
                    }
                    string CensoredText = ConfigKey.SwearReplace.GetString()+"&F";

                    const string PatternTemplate = @"\b({0})(s?)\b";
                    const RegexOptions Options = RegexOptions.IgnoreCase;

                    if (Swears.Count == 0)
                    {
                        Swears.AddRange(File.ReadAllLines("SwearWords.txt").Where(line => line.StartsWith("#") == false || line.Trim().Equals(String.Empty)));
                    }

                    if (badWordMatchers == null)
                    {
                        badWordMatchers = Swears.
                            Select(x => new Regex(string.Format(PatternTemplate, x), Options));
                    }

                    string output = badWordMatchers.
                       Aggregate(rawMessage, (current, matcher) => matcher.Replace(current, CensoredText));
                    rawMessage = output;
                }

                string formattedMessage = String.Format("{0}&F: {1}",
                                                         player.ClassyName,
                                                         rawMessage);

Re: Profanity filter

Posted: January 11th, 2012, 6:50 pm
by BobKare
Maybe if 3 letters or more match a swearing word, it will be cencored?

Like I add "fuck" to the list, then it'll cencore "fucking", "fukk", "fvck" etc...
If less than three characters matches, it won't be much of a bad word.
"fukc" will also be cencored, since the letters mustn't be in correct order.

Re: Profanity filter

Posted: January 11th, 2012, 7:50 pm
by Jonty800
I'll develop this more one day so players cant say things like fvck if fuck is in the list.
The code I used before this one made it so fuck also blocked out fucking, but there are a lot of problems to consider when using that method. When server hosts add "ass" to the list, then you end up with "My teacher set me an ********* today" (assignment). "Jimmy could you **** me the ball?" (pass)

I think fragmer was explaining how u looks like v, like l looks like I.
I could use a dictionary and add things like
.Add("o", "[0]");
.Add("u", "[v]");
Which will then read the text file and add new words to the cache (the words you add to the SwearWords.txt, but it replaces a with @, I with l... ect)

Re: Profanity filter

Posted: January 13th, 2012, 3:32 am
by Ollieboy
Nice work. You should go and start a flame war in Runescape to see what triggers their swearing and how people get around it :P

Re: Profanity filter

Posted: January 13th, 2012, 11:20 am
by fragmer
Another thing that can make filters more flexible is addition of Soundex matching. And more generally, approximate string matching and phonetic algorithms.

Those algorithms can be very CPU-intensive though, so I recommend caution. Not that I recommend/endorse profanity filtering at all, dont get me wrong :P

Re: Profanity filter

Posted: January 13th, 2012, 11:43 am
by Intertoothh

Re: Profanity filter

Posted: January 13th, 2012, 9:55 pm
by Jonty800
Thanks for the tips guys :) Nothing can be more cpu-intensive than Minecraft itself :P

I don't like profanity filtering either, it tarnishes Guest's anger when they get caught griefing haha!

Updated fCraft compatible code in the 1st post.

Re: Profanity filter

Posted: January 15th, 2016, 3:06 am
by adseo2016
Updated fCraft compatible code in the 1st post.
gclub