Bing [Bot]

Have something to say about forums.debian.net itself?

Bing [Bot]

Postby roseway » 2016-11-05 07:20

This is just idle curiosity on my part, but I noticed that, in the "Who is online" section, one of the registered users is shown as Bing [Bot]. How does a search engine bot get itself to show as a registered user?
Eric
User avatar
roseway
 
Posts: 1462
Joined: 2007-12-31 22:50
Location: Kent, UK

Re: Bing [Bot]

Postby GarryRicketson » 2016-11-05 13:30

Yes that is correct, the search bots are registered users, they have specific
permissions, and restrictions,....for example a search bot can not post anything.
So when a search bot is browsing, it shows as a registered user, "google",
"yahoo", "bing", etc......
The bot itself can not and does not make itself a registered user, that is done
by a forum admin, actually even in a default installation it is normal for the
phpbb to have permissions and roles set for search bots.
https://www.phpbb.com/community/viewtopic.php?f=46&t=1792705
User avatar
GarryRicketson
 
Posts: 3870
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

Postby roseway » 2016-11-05 18:30

I understand why specific permissions are enabled for search bots, but it baffles me why they need to be manually registered. Anyone can browse the forums without being registered, and that's what search bots normally do.

But as I said, this is only idle curiosity.
Eric
User avatar
roseway
 
Posts: 1462
Joined: 2007-12-31 22:50
Location: Kent, UK

Re: Bing [Bot]

Postby kedaha » 2016-11-06 16:13

I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.
User avatar
kedaha
 
Posts: 2589
Joined: 2008-05-24 12:26

Re: Bing [Bot]

Postby Bulkley » 2016-11-06 16:37

There's a bit of give and take with bots. The Bing bot, for example, should keep the Bing search engine up to date on the activity of this forum. A google bot should do the same for Google. Etc. I say should because I'm never quite sure that's all they do.
Bulkley
 
Posts: 5104
Joined: 2006-02-11 18:35

Re: Bing [Bot]

Postby GarryRicketson » 2016-11-06 18:05

Depending on who you ask, some people don't like them and do "ban" or
use the robots.txt to give instructions about their site to web robots.
There are "good" bots and "bad" bots, the "good" ones are usually obediant and respect any "instructions", the "bad" ones don't.

kedaha wrote:I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.


This is a list, that is used in the phpBB software,
Code: Select all
 Google [Bot]        
Yahoo [Bot]        
Bing [Bot]         
AdsBot [Google]     
Alexa [Bot]       
Alta Vista [Bot]     
Ask Jeeves [Bot]     
Baidu [Spider]        
Exabot [Bot]         
FAST Enterprise      
FAST WebCrawler     
Francis [Bot]        
Gigabot [Bot]        
Google Adsense [Bot]          
Google Desktop         
Google Feedfetcher        
Heise IT-Markt [Crawler]       
Heritrix [Crawler]        
IBM Research [Bot]         
ICCrawler - ICjobs       
MSN NewsBlogs        
MSN [Bot]       
MSNbot Media         
Majestic-12 [Bot]         
Metager [Bot]        
NG-Search [Bot]       
Nutch [Bot]       
Nutch/CVS [Bot]        
OmniExplorer [Bot]        
Online link [Validator]        
SEO Crawler         
SEOSearch [Crawler]         
Seekport [Bot]        
Sensis [Crawler]         
Seoma [Crawler]       
Snappy [Bot]        
Steeler [Crawler]          
Synoo [Bot]         
Telekom [Bot]        
TurnitinBot [Bot]        
Voyager [Bot]         
W3 [Sitesearch]         
W3C [Linkcheck]         
W3C [Validator]       
WiseNut [Bot]        
Yacy [Bot]       
Yahoo MMCrawler [Bot]    
Yahoo Slurp [Bot]        
YahooSeeker [Bot]        
ichiro [Crawler]          
psbot [Picsearch]


They can be edited, deleted , etc :
For example:
Code: Select all
Bot name    Last visit    Options    Mark
Google [Bot]     Mon Oct 31, 2016 4:58 pm      Deactivate      Edit      Delete

I got this from a different forum, one that I administer, On this forum I am not
a admin, and don't know , nor have access to how it is set up.
Post by roseway » but it baffles me why they need to be manually registered.

I am sorry, maybe I worded previous post poorly, they are not "manually" registered,
it is more of a "default" list, and it they are not listed as "registered" users, it is a seperate group. " Spiders/Robots ",
Manage bots
“Bots”, “spiders” or “crawlers” are automated agents most commonly used by search engines to update their databases. Since they rarely make proper use of sessions they can distort visitor counts, increase load and sometimes fail to index sites correctly. Here you can define a special type of user to overcome these problems.

And that is where it gets to be a "manual" process, when the admins need to "define"
special types of users, etc. Decide which bots to allow, and where they are allowed
to go, etc,......

There are some really "bad bots", that even go so far as to try to "look like "
the google bot, (for example), I think these are what give the "good bots" such a
bad reputation,
by Bulkley » I say should because I'm never quite sure that's all they do.

I think in many cases, and this happened to me, I thought it was a "google bot",
because that was what it said it was, but it was trying to change it's "profile" as
if it was a normal user, so my imediate reaction was to "ban" and block the google
bot, but later when I was asking about it, on another forum, I learned that it was not
a real google bot, the real google bot does not try to do that sort of thing.
Adminstrating a server/website, forums, etc. is much more complicated then
many people realize, it is important that the owner keeps track of what the "bots"
and visisitors are doing, if the site owner can not, or does not want to , then they
need to have administrators that can be trusted, and know how to deal with those
"tasks",... ..
I don't really know that much, in fact I just barely "scratch" the surface, all though
it is something that interests me, and I am constantly trying to learn more,....
User avatar
GarryRicketson
 
Posts: 3870
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

Postby 4D696B65 » 2016-11-06 22:30

kedaha wrote:I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.

We have 51 registered. Most log in as invisible.
User avatar
4D696B65
 
Posts: 1960
Joined: 2009-06-28 06:09

Re: Bing [Bot]

Postby millpond » 2016-11-15 05:33

Search engine bots are quite common on all public sites - and some sites go out of their way to invite them in and customize listings for them.

Myself, i block them at robots.txt and .htaccess, which of course means that searching for my site will come up with zero results.
The way I want it.

There are logical reasons why this site would want exposure to search engines, and it wouldnt make a difference how diabolocal they wrere. M$ is hardly, at heart, any worse than google.
All out to get control of your info and your desktop.
millpond
 
Posts: 582
Joined: 2014-06-25 04:56

Re: Bing [Bot]

Postby tomazzi » 2016-11-16 00:28

OK, so we have 51 "spies" - but this is not a reply for my original question: (re-formed):
So, what is the BingBot doing here - listed as a registered user?

If the code of this bot requires listing as a "normal" user - then it simply means a bug in the code - and if this is true (personally I doubt it) - then such bot should be simply kicked-out (as a buggy software)

Another thing is, that Google, whether You like it or not (I don't), has ~75-80% of the search engine market share, so who cares about Bing?

... Especially that Bing was proven to steal search results from Google... (I'm prepared to provide a link - max few seconds of searching)

... And additionally, the company which stays behind the Bing (MSFT) is a long-time declared foe of Debian or Linux in general -> i.e. this is a conflict of interests.

Regards.
Odi profanum vulgus
tomazzi
 
Posts: 730
Joined: 2013-08-02 21:33

Re: Bing [Bot]

Postby tomazzi » 2016-11-21 23:58

No reply is a reply - it means that debian forums is not an independant organization - it actually shows that the debian *is* dependant on external companies - and all the debian project declarations regarding the care of user's experience/rights are worth shit. No surprise here - the way in which the adoption of systemd made by RedHat was executed was just a tip of the iceberg.

Now I've learned something: Debian is just a puppet - it does not have any targets, especially after Ian have
*accidencially* died - and the users of the famous, stable "debian" are now on their own.

No reply is a reply - sometimes it's even better than having a "official" reply from a clueless puppet...

Anyway, regards.

Edit:
If the above is not clear, I'm just asking to be banned - literally and practically - please delete my account on this forums...
I know that You can't do this (freedom?) - You can only rename my account to something like "xxx-account-deleted" - so please do so...

Again, Regards...
Odi profanum vulgus
tomazzi
 
Posts: 730
Joined: 2013-08-02 21:33

Re: Bing [Bot]

Postby GarryRicketson » 2016-11-24 19:43

The question did get answered, the bing bot, is doing the same thing all
the other search engine bots are doing, getting data to add to the search
engine,.........
But in any event, the proper proceedure to get a user account removed,
or with any other account problems, is explained here :

HOWTO contact forum moderators/admins

If you have problems with accounts or with posting, or having any other technical problem or question, please contact admin@forums.debian.net. The same repeat-guideline as for team@ exists here.


Thank you,
from: "from a clueless puppet"
Last edited by GarryRicketson on 2016-11-24 23:01, edited 1 time in total.
User avatar
GarryRicketson
 
Posts: 3870
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

Postby Head_on_a_Stick » 2016-11-24 19:49

@tomazzi, what does the presence of a bot on these boards have to do with debian.org?

AFAIUI, these forums are independent and not officially part of the Debian project.
“Controlling complexity is the essence of computer programming."Brian Kernighan

Please read before posting How to report a problem
User avatar
Head_on_a_Stick
 
Posts: 6489
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Bing [Bot]

Postby dasein » 2016-11-24 22:55

tomazzi wrote:... who cares about Bing?

According to your own "estimate," one searcher in every five.

IMO, banning a(ny) legitimate search engine spider would violate DFSG.

tomazzi wrote: please delete my account on this forums...

Absolutely no need for melodrama. If you're determined to leave, simply change your password blindly (not looking at the screen or keyboard) to a bunch of random keypresses. You'll be just as gone, without imposing a pointless and unnecessary burden on the staff.
User avatar
dasein
 
Posts: 7378
Joined: 2011-03-04 01:06
Location: Terra Incantationum


Return to Forum stuff & feedback

Who is online

Users browsing this forum: No registered users and 1 guest

fashionable