Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

New Anti-Spam measures

Code of conduct, suggestions, and information on forums.debian.net.
Message
Author
Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: New Anti-Spam measures

#31 Post by Bulkley »

Randicus wrote:Ethnic IP banning is a phrase I created for the post. I chose it, because many blacklists are not rigorous in sorting spam from real users. In effect, they ban IP addresses from certain countries. I am not claiming the ones used here do that, but this kind of measure inevitably involves some degree of it. I mentioned the problem with blacklisting IPs simply to draw attention to a potential problem. I do not want members thinking it is a panacea to the spam woes.
The better spam checkers, such as stopforumspam, don't blacklist on the basis of IP alone. They use a combination of IP, email address and user name. Their database is built on confirmed complaints and is very reliable.

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: New Anti-Spam measures

#32 Post by dasein »

Bulkley wrote:They use a combination of IP, email address and user name.
The really smart move would be to blacklist based primarily on spamvertised URL. URL blacklisting has a proven history of effectiveness against email spam, and has very few false positives. The flip side is that it's probably more computationally expensive.

Randicus
Posts: 2663
Joined: 2011-05-08 09:11

Re: New Anti-Spam measures

#33 Post by Randicus »

Bulkley wrote:The better spam checkers, such as stopforumspam, don't blacklist on the basis of IP alone. They use a combination of IP, email address and user name. Their database is built on confirmed complaints and is very reliable.
Hopefully it works as well as it sounds. What about the other one? Honeypot?
And rupeshforu? According to Mez, he was flagged as a spammer, because of his service provider, not because his name was entered onto a list of spammers.
Hi there,

Just had a little look into why you're being flagged as a spammer.

You're currently using a Dynamic IP handed out from Vodafone India.

This IP has previously been caught being used by an automatic spam-bot (it got caught in what is known as a honeypot).

This will only affect registration or changing your profile while you are still on that IP address.
I believe this incident was the day after the change was implemented.
arochester wrote:There is a mis-quote wrongly attributed to Abraham Lincoln
You can please some of the people all of the time, and all of the people some of the time, but you can not please all of the people all of the time.
The issue is not about pleasing anyone. It is a case of cost-benefit analysis. The most benefit with the least drawback. Or as you put it:
Sometimes, general measures are for the good of the majority and will catch out some innocent individuals. Does that mean the general measures should not be put in place?
The question then is; Does the benefit of removing a great amount of spam (but not all) out-weigh the disadvantage of a smaller number of potential members not being able to register?
That of course will be a point of debate between members with different views. For my part, I am reminded of another famous quote.
I would rather have a hundred criminals go free than one innocent person go to prison.
I know many of the board's members will not understand my concern. Because they live in Europe and North America, they do not know what is like to be the victim of such measures. Believe me, it is very frustrating. As much as I would like to see the spam quashed, I do not like the idea of one potential new member being denied for every ten, twenty or hundred spammers denied. Simply put, my position on the scale of cost-benefit analysis is influenced by my experience.
On balance are people happy or unhappy with direction of the changes?
To reiterate what I stated in an earlier post, my intention was not to attack the mighty Mez, campaign for the right of spammers or to make myself look good. I simply want members to be aware of a potential (and very likely) problem that I believe deserves consideration. My concern is for the health and future of the board. Both for current and future members. (And potential future members.)

vbrummond
Posts: 4432
Joined: 2010-03-02 01:42

Re: New Anti-Spam measures

#34 Post by vbrummond »

:o
Last edited by vbrummond on 2013-03-26 06:30, edited 1 time in total.
Always on Debian Testing

User avatar
Absent Minded
Posts: 3464
Joined: 2006-07-09 08:50
Location: Washington State U.S.A.
Been thanked: 3 times

Re: New Anti-Spam measures

#35 Post by Absent Minded »

Randicus wrote:......
The question then is; Does the benefit of removing a great amount of spam (but not all) out-weigh the disadvantage of a smaller number of potential members not being able to register?
That of course will be a point of debate between members with different views. For my part, I am reminded of another famous quote.
I would rather have a hundred criminals go free than one innocent person go to prison.
I know many of the board's members will not understand my concern. Because they live in Europe and North America, they do not know what is like to be the victim of such measures. Believe me, it is very frustrating. As much as I would like to see the spam quashed, I do not like the idea of one potential new member being denied for every ten, twenty or hundred spammers denied. Simply put, my position on the scale of cost-benefit analysis is influenced by my experience.
On balance are people happy or unhappy with direction of the changes?
To reiterate what I stated in an earlier post, my intention was not to attack the mighty Mez, campaign for the right of spammers or to make myself look good. I simply want members to be aware of a potential (and very likely) problem that I believe deserves consideration. My concern is for the health and future of the board. Both for current and future members. (And potential future members.)
Just to be fair about things and be clear, we are not talking about 1 person in every 100 or even 1000. In less than 24 hours these new measures blocked over 20,000 spam attempts. Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation. So your analigy doesn't really work in this situation.

I wonder if they use Debian on the Enterprise? To quote a little SiFi logic, "The needs of the many out-weigh the needs of the few or the one." This can also be revirsed in some cases.

AM
Serving the community the best way I can.
Spreading the tradition of Community Spirit.
Please read some Basic Forum Philosophy
Give a man a fish, he eats for a day. Teach him how to fish, he eats for life.
Updated Nov. 19, 2012

Randicus
Posts: 2663
Joined: 2011-05-08 09:11

Re: New Anti-Spam measures

#36 Post by Randicus »

Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.
If so, that would greatly reduce the chances of people being locked out. I am glad to know such a measure has been taken. If people who may be adversely affected have an option to easily resolve it, that is good. Hopefully my concern can be laid aside.

I shall not comment on the attitude put forth by more than one person concerning the few who could have been/are/would have been affected.

AnInkedSoul
Posts: 461
Joined: 2010-06-11 05:05

Re: New Anti-Spam measures

#37 Post by AnInkedSoul »

Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.
Actually it does not seem to tell you how to contact the staff, you are just told the following on the registration page.
Your IP ------ or your username ----- or your e-mail address --------- has been blocked because it is blacklisted. For details please see http://www.stopforumspam.com/api?------ ... ----------
An entry on the blacklist may have several reasons:
1. You are a well-known spammer.
2. Last time a well-known spammer was using the dynamic IP address which you got from your ISP (Internet Service Provider), your e-mail address or the username you have choosen.
3. Your ISP is well-known for a lot of spamming customers and is not fighting against spammers enough.
Oh and that last reason seems kinda shitty to me.

Just sayin...

User avatar
llivv
Posts: 5340
Joined: 2007-02-14 18:10
Location: cold storage

Re: New Anti-Spam measures

#38 Post by llivv »

I guess it ain't prefect yet, oh well ....grrrrrrrrr
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.

confuseling
Posts: 2121
Joined: 2009-10-21 01:03

Re: New Anti-Spam measures

#39 Post by confuseling »

I'd be more convinced if the people complaining about unfair profiling were spamhunters.

Just saying...
The Forum's search box is terrible. Use site specific search, e.g.
https://www.google.com/search?q=site%3A ... terms+here

Randicus
Posts: 2663
Joined: 2011-05-08 09:11

Re: New Anti-Spam measures

#40 Post by Randicus »

Who is complaining?
I pointed out a potential problem. Concern is not complaining.
AnInkedSoul provided information showing that the safeguard Absent Minded mentioned does not follow the process attributed to it. That is a clarification, not a complaint.

Anyway. I gave up on it. I only made this post to point out that raising a concern is not complaining. I do not know about AnInkedSoul, but if I would have complained, everyone would have known it.

User avatar
Absent Minded
Posts: 3464
Joined: 2006-07-09 08:50
Location: Washington State U.S.A.
Been thanked: 3 times

Re: New Anti-Spam measures

#41 Post by Absent Minded »

AnInkedSoul wrote:
Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.
Actually it does not seem to tell you how to contact the staff, you are just told the following on the registration page.
Your IP ------ or your username ----- or your e-mail address --------- has been blocked because it is blacklisted. For details please see http://www.stopforumspam.com/api?------ ... ----------
An entry on the blacklist may have several reasons:
1. You are a well-known spammer.
2. Last time a well-known spammer was using the dynamic IP address which you got from your ISP (Internet Service Provider), your e-mail address or the username you have choosen.
3. Your ISP is well-known for a lot of spamming customers and is not fighting against spammers enough.
Oh and that last reason seems kinda shitty to me.

Just sayin...
I am sure the contact info was suppose to be on the reason for denial information page. I will ask Mez about it though just incase somehow it has been missed. I don't know the url of the page so I can look at it myself.

While it would really suck to be an innocent victom of a rogue ISP, denying their entire IP range is about the only way to deal with ISPs that refuse to play ball with the rest of the internet. Once your customers can't go anywhere, they complain, then they change services if nothing changes. Loss of revinew is about the only thing some companies care about and the only way to get them to play nice with the rest of us on the internet.
Serving the community the best way I can.
Spreading the tradition of Community Spirit.
Please read some Basic Forum Philosophy
Give a man a fish, he eats for a day. Teach him how to fish, he eats for life.
Updated Nov. 19, 2012

MartinTruckenbrodt
Posts: 1
Joined: 2013-04-21 19:54

Re: New Anti-Spam measures

#42 Post by MartinTruckenbrodt »

Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin

User avatar
Absent Minded
Posts: 3464
Joined: 2006-07-09 08:50
Location: Washington State U.S.A.
Been thanked: 3 times

Re: New Anti-Spam measures

#43 Post by Absent Minded »

Hi Martin, thank you for stopping by. I will pass this post onto our server admins so they can have a look at it.
Serving the community the best way I can.
Spreading the tradition of Community Spirit.
Please read some Basic Forum Philosophy
Give a man a fish, he eats for a day. Teach him how to fish, he eats for life.
Updated Nov. 19, 2012

User avatar
Mez
Debian Developer, Forum Ninja
Debian Developer, Forum Ninja
Posts: 112
Joined: 2009-04-23 22:04

Re: New Anti-Spam measures

#44 Post by Mez »

MartinTruckenbrodt wrote:Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin
Hi Martin,

That's what we're using :)

Thanks for your work!

EDIT: Current stats: 632,456 spam attempts (automatically) blocked since we've put in place the new measures. We've also contributed back 124,080 entries to the centralised database.

User avatar
ComputerBob
Posts: 1181
Joined: 2007-11-30 04:49
Location: The Mountains of the Sunshine State
Been thanked: 1 time

Re: New Anti-Spam measures

#45 Post by ComputerBob »

Mez wrote:
MartinTruckenbrodt wrote:Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin
Hi Martin,

That's what we're using :)

Thanks for your work!

EDIT: Current stats: 632,456 spam attempts (automatically) blocked since we've put in place the new measures. We've also contributed back 124,080 entries to the centralised database.
Wow! Thanks for that excellent news! 8)
ComputerBob - Making Geek-Speak Chic (TM)
ComputerBob.com - Nearly 6,000 Posts and 23 Million Views
My Massive Stroke
Help! (off-topic)
_________________
Your Life Matters

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: New Anti-Spam measures

#46 Post by dasein »

Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.

User avatar
Mez
Debian Developer, Forum Ninja
Debian Developer, Forum Ninja
Posts: 112
Joined: 2009-04-23 22:04

Re: New Anti-Spam measures

#47 Post by Mez »

dasein wrote:Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.
Well, spam "hits". Bots will generally go throguh a process of a few pages before the spam is actually posted (register, update profile, post spam) etc etc - the smart bots will check whether their action is successful - but for the most part - the bots aren't smart - and will just spam the actions at the server - hoping that they're getting through.

So the number given is attempted spam "hits" - whether that be as the initial part of the process or not.

Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: New Anti-Spam measures

#48 Post by Bulkley »

dasein wrote:Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.
I believe it. I help moderate another website and it gets hammered. It's bots that do it. As Mez said, the number is attempts, not successes. We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: New Anti-Spam measures

#49 Post by dasein »

Bulkley wrote:We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.
But we didn't have an automatic anti-spam widget in place a month or so ago,and I was part of the small group who fought the problem manually. I can't speak to other venues, but I am confident that FDN did not have anywhere near 17,000 spams per day, not even on the worst day we ever had. It's off by at least one order of magnitude.

The "overcounting" explanation makes much more sense.

Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: New Anti-Spam measures

#50 Post by Bulkley »

dasein wrote:
Bulkley wrote:We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.
But we didn't have an automatic anti-spam widget in place a month or so ago,and I was part of the small group who fought the problem manually. I can't speak to other venues, but I am confident that FDN did not have anywhere near 17,000 spams per day, not even on the worst day we ever had. It's off by at least one order of magnitude.
You dealt with what got by the Captcha. Bots keep trying until they get through. Attempts will always outnumber successes.

Post Reply