It is currently December 13th, 2017, 6:48 pm


All times are UTC




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: Profanity filter
PostPosted: January 11th, 2012, 12:40 am 

Joined: August 21st, 2011, 6:31 am
Posts: 280
Location: United Kingdom
Swearing on servers can be annoying, so I thought I'd share this.

Its a simple profanity filter that I wrote. It works by replacing any bad words with a word of your choosing.
The filter works by replacing full words, that way people can still say words like "Scunthorpe"

1. Add a permission called "Swear" in the Permissions.cs
2. Go to chat.cs. Add these two at the top of the page.
Code:
using System.Text.RegularExpressions;
using System.IO;

Add just above public static bool SendGlobal:
Code:
public static List<string> Swears = new List<string>();
        public static IEnumerable<Regex> badWordMatchers;

Then you can add this into SendGlobal (Caution: contains swears)

Show
Code:
if (!player.Can(Permission.Swear))
                {
                    if (!File.Exists("SwearWords.txt"))
                    {
                        StringBuilder sb = new StringBuilder();
                        sb.AppendLine("#This txt file should be filled with bad words that you want to be filtered out");
                        sb.AppendLine("#I have included some examples, excuse my language :P");
                        sb.AppendLine("fuck");
                        sb.AppendLine("fucking");
                        sb.AppendLine("fucked");
                        sb.AppendLine("dick");
                        sb.AppendLine("bitch");
                        sb.AppendLine("shit");
                        sb.AppendLine("shitting");
                        sb.AppendLine("shithead");
                        sb.AppendLine("cunt");
                        sb.AppendLine("nigger");
                        sb.AppendLine("wanker");
                        sb.AppendLine("wank");
                        sb.AppendLine("wanking");
                        sb.AppendLine("piss");
                        File.WriteAllText("SwearWords.txt", sb.ToString());
                    }
                    string CensoredText = Color.ReplacePercentCodes("&CBlock&F"); //you can change this. Always end in &F though
                   

                    const string PatternTemplate = @"\b({0})(s?)\b";
                    const RegexOptions Options = RegexOptions.IgnoreCase;

                    if (Swears.Count == 0)
                    {
                        Swears.AddRange(File.ReadAllLines("SwearWords.txt").Where(line => line.StartsWith("#") == false || line.Trim().Equals(String.Empty)));
                    }

                    if (badWordMatchers == null)
                    {
                        badWordMatchers = Swears.
                            Select(x => new Regex(string.Format(PatternTemplate, x), Options));
                    }

                    string output = badWordMatchers.
                       Aggregate(rawMessage, (current, matcher) => matcher.Replace(current, CensoredText));
                    rawMessage = output;
                }
                }


3. You can replace string CensoredText with whatever word you like, so if i swear it will now say "block off you block". Feel free to use color codes too, as long as the string ends with %F or has '+"&F";' at the end.

4. In the configGUI, go to ranks and then you can tick "Swear" to each rank that you would like to be able to swear on the server. (Note the filter only works in general chat, players can still swear in staffchat and pm's.

5. When you start the server, when someone firsts talks, a new file in the root of your server will be made called "SwearWords.txt". When the server is shutdown, you can edit this file adding new swear words that you wish to be filtered. Make sure these are on new lines.
Since the filter works using full words, make sure you add things like f**k and f**king (ignore the asterixis). A list of default words will be added when the file is first made.

_________________
You cannot use certain BBCodes: [img].


Last edited by Jonty800 on January 20th, 2012, 6:40 pm, edited 2 times in total.

Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 1:52 am 
fCraft Developer
User avatar

Joined: May 21st, 2011, 10:53 pm
Posts: 1386
This will work, but I'm worried about performance. Current code does disk i/o and may create hundreds of temporary objects for each message. I'd cache the list and especially the Regex objects.

Also keep in mind that the whole chat-censorship effort is completely futile. If you censor "fuck" people will still write "fvck", or "fukk", etc.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 2:52 am 

Joined: August 21st, 2011, 6:31 am
Posts: 280
Location: United Kingdom
so would loading the profanity list on start-up solve this? So when the server starts, the SwearWords.txt is placed in a cache so the list doesnt need to be loaded again?

edit: nvm I just used a list

_________________
You cannot use certain BBCodes: [img].


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 3:28 am 
Trustee
User avatar

Joined: May 26th, 2011, 6:35 pm
Posts: 324
I would use it to replace "pony", "mlp", etc with very bad puns.

_________________
<FCB2> trevbev94: who made the perimid out of saind cause that is ausom
<FCB2> Pemalite+: druffin. 'cmere big boy for some lovin'


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 3:38 am 

Joined: August 21st, 2011, 6:31 am
Posts: 280
Location: United Kingdom
I made a cache for the list and for IEnumerable<Regex> badWordMatchers

I can't see anything else to cache. Does this look ok?

Show
Code:
Top of the page:
public static List<string> Swears = new List<string>();
        public static IEnumerable<Regex> badWordMatchers;

In SendGlobal:

if (!player.Can(Permission.Swear))
                {
                    if (!File.Exists("SwearWords.txt"))
                    {
                        StringBuilder sb = new StringBuilder();
                        sb.AppendLine("#This txt file should be filled with bad words that you want to be filtered out");
                        sb.AppendLine("#I have included some examples, excuse my language :P");
                        sb.AppendLine("fuck");
                        sb.AppendLine("fucking");
                        sb.AppendLine("fucked");
                        sb.AppendLine("dick");
                        sb.AppendLine("bitch");
                        sb.AppendLine("shit");
                        sb.AppendLine("shitting");
                        sb.AppendLine("shithead");
                        sb.AppendLine("cunt");
                        sb.AppendLine("nigger");
                        sb.AppendLine("wanker");
                        sb.AppendLine("wank");
                        sb.AppendLine("wanking");
                        sb.AppendLine("dick");
                        sb.AppendLine("piss");
                        File.WriteAllText("SwearWords.txt", sb.ToString());
                    }
                    string CensoredText = ConfigKey.SwearReplace.GetString()+"&F";

                    const string PatternTemplate = @"\b({0})(s?)\b";
                    const RegexOptions Options = RegexOptions.IgnoreCase;

                    if (Swears.Count == 0)
                    {
                        Swears.AddRange(File.ReadAllLines("SwearWords.txt").Where(line => line.StartsWith("#") == false || line.Trim().Equals(String.Empty)));
                    }

                    if (badWordMatchers == null)
                    {
                        badWordMatchers = Swears.
                            Select(x => new Regex(string.Format(PatternTemplate, x), Options));
                    }

                    string output = badWordMatchers.
                       Aggregate(rawMessage, (current, matcher) => matcher.Replace(current, CensoredText));
                    rawMessage = output;
                }

                string formattedMessage = String.Format("{0}&F: {1}",
                                                         player.ClassyName,
                                                         rawMessage);

_________________
You cannot use certain BBCodes: [img].


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 6:50 pm 

Joined: May 26th, 2011, 2:15 pm
Posts: 279
Maybe if 3 letters or more match a swearing word, it will be cencored?

Like I add "fuck" to the list, then it'll cencore "fucking", "fukk", "fvck" etc...
If less than three characters matches, it won't be much of a bad word.
"fukc" will also be cencored, since the letters mustn't be in correct order.


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 11th, 2012, 7:50 pm 

Joined: August 21st, 2011, 6:31 am
Posts: 280
Location: United Kingdom
I'll develop this more one day so players cant say things like fvck if fuck is in the list.
The code I used before this one made it so fuck also blocked out fucking, but there are a lot of problems to consider when using that method. When server hosts add "ass" to the list, then you end up with "My teacher set me an ********* today" (assignment). "Jimmy could you **** me the ball?" (pass)

I think fragmer was explaining how u looks like v, like l looks like I.
I could use a dictionary and add things like
.Add("o", "[0]");
.Add("u", "[v]");
Which will then read the text file and add new words to the cache (the words you add to the SwearWords.txt, but it replaces a with @, I with l... ect)

_________________
You cannot use certain BBCodes: [img].


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 13th, 2012, 3:32 am 
Trustee
User avatar

Joined: May 23rd, 2011, 3:13 am
Posts: 2341
Location: Australia
Nice work. You should go and start a flame war in Runescape to see what triggers their swearing and how people get around it :P

_________________
<TKB> Hit_Girl: zombies don't hurt
<TKB> Hit_Girl: weird.
<TKB> Hit_Girl was slain by Zombie


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 13th, 2012, 11:20 am 
fCraft Developer
User avatar

Joined: May 21st, 2011, 10:53 pm
Posts: 1386
Another thing that can make filters more flexible is addition of Soundex matching. And more generally, approximate string matching and phonetic algorithms.

Those algorithms can be very CPU-intensive though, so I recommend caution. Not that I recommend/endorse profanity filtering at all, dont get me wrong :P


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 13th, 2012, 11:43 am 
Trustee
User avatar

Joined: May 24th, 2011, 5:51 am
Posts: 1148
There are more options:
Metaphone: http://en.wikipedia.org/wiki/Metaphone
and
Levenshtein: http://en.wikipedia.org/wiki/Levenshtein_distance

_________________
McLaughlinKid wrote:
You put roar on everything don't you?


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 13th, 2012, 9:55 pm 

Joined: August 21st, 2011, 6:31 am
Posts: 280
Location: United Kingdom
Thanks for the tips guys :) Nothing can be more cpu-intensive than Minecraft itself :P

I don't like profanity filtering either, it tarnishes Guest's anger when they get caught griefing haha!

Updated fCraft compatible code in the 1st post.

_________________
You cannot use certain BBCodes: [img].


Top
Offline Profile  
Reply with quote  
 Post subject: Re: Profanity filter
PostPosted: January 15th, 2016, 3:06 am 

Joined: January 15th, 2016, 3:05 am
Posts: 1
Updated fCraft compatible code in the 1st post.
gclub


Top
Offline Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
Theme created StylerBB.net