Debian User Forums

Randicus wrote:Ethnic IP banning is a phrase I created for the post. I chose it, because many blacklists are not rigorous in sorting spam from real users. In effect, they ban IP addresses from certain countries. I am not claiming the ones used here do that, but this kind of measure inevitably involves some degree of it. I mentioned the problem with blacklisting IPs simply to draw attention to a potential problem. I do not want members thinking it is a panacea to the spam woes.

The better spam checkers, such as stopforumspam, don't blacklist on the basis of IP alone. They use a combination of IP, email address and user name. Their database is built on confirmed complaints and is very reliable.

Bulkley wrote:They use a combination of IP, email address and user name.

The really smart move would be to blacklist based primarily on spamvertised URL. URL blacklisting has a proven history of effectiveness against email spam, and has very few false positives. The flip side is that it's probably more computationally expensive.

Bulkley wrote:The better spam checkers, such as stopforumspam, don't blacklist on the basis of IP alone. They use a combination of IP, email address and user name. Their database is built on confirmed complaints and is very reliable.

Hopefully it works as well as it sounds. What about the other one? Honeypot?
And rupeshforu? According to Mez, he was flagged as a spammer, because of his service provider, not because his name was entered onto a list of spammers.

Hi there,

Just had a little look into why you're being flagged as a spammer.

You're currently using a Dynamic IP handed out from Vodafone India.

This IP has previously been caught being used by an automatic spam-bot (it got caught in what is known as a honeypot).

This will only affect registration or changing your profile while you are still on that IP address.

I believe this incident was the day after the change was implemented.

arochester wrote:There is a mis-quote wrongly attributed to Abraham Lincoln

You can please some of the people all of the time, and all of the people some of the time, but you can not please all of the people all of the time.

The issue is not about pleasing anyone. It is a case of cost-benefit analysis. The most benefit with the least drawback. Or as you put it:

Sometimes, general measures are for the good of the majority and will catch out some innocent individuals. Does that mean the general measures should not be put in place?

The question then is; Does the benefit of removing a great amount of spam (but not all) out-weigh the disadvantage of a smaller number of potential members not being able to register?
That of course will be a point of debate between members with different views. For my part, I am reminded of another famous quote.

I would rather have a hundred criminals go free than one innocent person go to prison.

I know many of the board's members will not understand my concern. Because they live in Europe and North America, they do not know what is like to be the victim of such measures. Believe me, it is very frustrating. As much as I would like to see the spam quashed, I do not like the idea of one potential new member being denied for every ten, twenty or hundred spammers denied. Simply put, my position on the scale of cost-benefit analysis is influenced by my experience.

On balance are people happy or unhappy with direction of the changes?

To reiterate what I stated in an earlier post, my intention was not to attack the mighty Mez, campaign for the right of spammers or to make myself look good. I simply want members to be aware of a potential (and very likely) problem that I believe deserves consideration. My concern is for the health and future of the board. Both for current and future members. (And potential future members.)

Randicus wrote:......
The question then is; Does the benefit of removing a great amount of spam (but not all) out-weigh the disadvantage of a smaller number of potential members not being able to register?
That of course will be a point of debate between members with different views. For my part, I am reminded of another famous quote.

I would rather have a hundred criminals go free than one innocent person go to prison.

I know many of the board's members will not understand my concern. Because they live in Europe and North America, they do not know what is like to be the victim of such measures. Believe me, it is very frustrating. As much as I would like to see the spam quashed, I do not like the idea of one potential new member being denied for every ten, twenty or hundred spammers denied. Simply put, my position on the scale of cost-benefit analysis is influenced by my experience.

On balance are people happy or unhappy with direction of the changes?

To reiterate what I stated in an earlier post, my intention was not to attack the mighty Mez, campaign for the right of spammers or to make myself look good. I simply want members to be aware of a potential (and very likely) problem that I believe deserves consideration. My concern is for the health and future of the board. Both for current and future members. (And potential future members.)

Just to be fair about things and be clear, we are not talking about 1 person in every 100 or even 1000. In less than 24 hours these new measures blocked over 20,000 spam attempts. Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation. So your analigy doesn't really work in this situation.

I wonder if they use Debian on the Enterprise? To quote a little SiFi logic, "The needs of the many out-weigh the needs of the few or the one." This can also be revirsed in some cases.

AM

Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.

If so, that would greatly reduce the chances of people being locked out. I am glad to know such a measure has been taken. If people who may be adversely affected have an option to easily resolve it, that is good. Hopefully my concern can be laid aside.

I shall not comment on the attitude put forth by more than one person concerning the few who could have been/are/would have been affected.

Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.

Actually it does not seem to tell you how to contact the staff, you are just told the following on the registration page.

Your IP ------ or your username ----- or your e-mail address --------- has been blocked because it is blacklisted. For details please see http://www.stopforumspam.com/api?------ ... ----------
An entry on the blacklist may have several reasons:
1. You are a well-known spammer.
2. Last time a well-known spammer was using the dynamic IP address which you got from your ISP (Internet Service Provider), your e-mail address or the username you have choosen.
3. Your ISP is well-known for a lot of spamming customers and is not fighting against spammers enough.

Oh and that last reason seems kinda shitty to me.

Just sayin...

I guess it ain't prefect yet, oh well ....grrrrrrrrr

I'd be more convinced if the people complaining about unfair profiling were spamhunters.

Just saying...

Who is complaining?
I pointed out a potential problem. Concern is not complaining.
AnInkedSoul provided information showing that the safeguard Absent Minded mentioned does not follow the process attributed to it. That is a clarification, not a complaint.

Anyway. I gave up on it. I only made this post to point out that raising a concern is not complaining. I do not know about AnInkedSoul, but if I would have complained, everyone would have known it.

AnInkedSoul wrote:

Absent Minded wrote:Additionally, if someone is not able to register they are redirected to a page that tells them how to contact our staff to remidy the problem. So, nobody is being locked out with out a remidy to fix the situation.

Actually it does not seem to tell you how to contact the staff, you are just told the following on the registration page.

Your IP ------ or your username ----- or your e-mail address --------- has been blocked because it is blacklisted. For details please see http://www.stopforumspam.com/api?------ ... ----------
An entry on the blacklist may have several reasons:
1. You are a well-known spammer.
2. Last time a well-known spammer was using the dynamic IP address which you got from your ISP (Internet Service Provider), your e-mail address or the username you have choosen.
3. Your ISP is well-known for a lot of spamming customers and is not fighting against spammers enough.

Oh and that last reason seems kinda shitty to me.

Just sayin...

I am sure the contact info was suppose to be on the reason for denial information page. I will ask Mez about it though just incase somehow it has been missed. I don't know the url of the page so I can look at it myself.

While it would really suck to be an innocent victom of a rogue ISP, denying their entire IP range is about the only way to deal with ISPs that refuse to play ball with the rest of the internet. Once your customers can't go anywhere, they complain, then they change services if nothing changes. Loss of revinew is about the only thing some companies care about and the only way to get them to play nice with the rest of us on the internet.

Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin

Hi Martin, thank you for stopping by. I will pass this post onto our server admins so they can have a look at it.

MartinTruckenbrodt wrote:Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin

Hi Martin,

That's what we're using

Thanks for your work!

EDIT: Current stats: 632,456 spam attempts (automatically) blocked since we've put in place the new measures. We've also contributed back 124,080 entries to the centralised database.

Mez wrote:

MartinTruckenbrodt wrote:Hello,
you have a very interesting discussion about spam prevention here. ...

I'm the MOD author of Advanced Block MOD for phpBB3. It's a very effective solution for preventing spam with blacklists. It reduces false positives to a minimum. In my experience it reduces it to Zero. With Advanced Block MOD you can disable all CAPTCHAs completely. So it's very user fiendly, too.

Bye Martin

Hi Martin,

That's what we're using

Thanks for your work!

EDIT: Current stats: 632,456 spam attempts (automatically) blocked since we've put in place the new measures. We've also contributed back 124,080 entries to the centralised database.

Wow! Thanks for that excellent news!

Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.

dasein wrote:Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.

Well, spam "hits". Bots will generally go throguh a process of a few pages before the spam is actually posted (register, update profile, post spam) etc etc - the smart bots will check whether their action is successful - but for the most part - the bots aren't smart - and will just spam the actions at the server - hoping that they're getting through.

So the number given is attempted spam "hits" - whether that be as the initial part of the process or not.

dasein wrote:Over 17,000 actual, honest-to-gawd spam attempts per day????

Seems unlikely.

I believe it. I help moderate another website and it gets hammered. It's bots that do it. As Mez said, the number is attempts, not successes. We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.

Bulkley wrote:We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.

But we didn't have an automatic anti-spam widget in place a month or so ago,and I was part of the small group who fought the problem manually. I can't speak to other venues, but I am confident that FDN did not have anywhere near 17,000 spams per day, not even on the worst day we ever had. It's off by at least one order of magnitude.

The "overcounting" explanation makes much more sense.

dasein wrote:

Bulkley wrote:We would be buried if we did not have a good anti-spam mod. The one that has been added to Debian User Forums has made a big improvement.

But we didn't have an automatic anti-spam widget in place a month or so ago,and I was part of the small group who fought the problem manually. I can't speak to other venues, but I am confident that FDN did not have anywhere near 17,000 spams per day, not even on the worst day we ever had. It's off by at least one order of magnitude.

You dealt with what got by the Captcha. Bots keep trying until they get through. Attempts will always outnumber successes.