Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Bing [Bot]

Code of conduct, suggestions, and information on forums.debian.net.
Post Reply
Message
Author
User avatar
roseway
Posts: 1528
Joined: 2007-12-31 22:50
Location: Kent, UK
Has thanked: 3 times
Been thanked: 4 times

Bing [Bot]

#1 Post by roseway »

This is just idle curiosity on my part, but I noticed that, in the "Who is online" section, one of the registered users is shown as Bing [Bot]. How does a search engine bot get itself to show as a registered user?
Eric

User avatar
GarryRicketson
Posts: 5644
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

#2 Post by GarryRicketson »

Yes that is correct, the search bots are registered users, they have specific
permissions, and restrictions,....for example a search bot can not post anything.
So when a search bot is browsing, it shows as a registered user, "google",
"yahoo", "bing", etc......
The bot itself can not and does not make itself a registered user, that is done
by a forum admin, actually even in a default installation it is normal for the
phpbb to have permissions and roles set for search bots.
https://www.phpbb.com/community/viewtop ... &t=1792705

User avatar
roseway
Posts: 1528
Joined: 2007-12-31 22:50
Location: Kent, UK
Has thanked: 3 times
Been thanked: 4 times

Re: Bing [Bot]

#3 Post by roseway »

I understand why specific permissions are enabled for search bots, but it baffles me why they need to be manually registered. Anyone can browse the forums without being registered, and that's what search bots normally do.

But as I said, this is only idle curiosity.
Eric

kedaha
Posts: 3521
Joined: 2008-05-24 12:26
Has thanked: 33 times
Been thanked: 77 times

Re: Bing [Bot]

#4 Post by kedaha »

I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.
DebianStable

Code: Select all

$ vrms

No non-free or contrib packages installed on debian!  rms would be proud.

Bulkley
Posts: 6383
Joined: 2006-02-11 18:35
Has thanked: 2 times
Been thanked: 39 times

Re: Bing [Bot]

#5 Post by Bulkley »

There's a bit of give and take with bots. The Bing bot, for example, should keep the Bing search engine up to date on the activity of this forum. A google bot should do the same for Google. Etc. I say should because I'm never quite sure that's all they do.

User avatar
GarryRicketson
Posts: 5644
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

#6 Post by GarryRicketson »

Depending on who you ask, some people don't like them and do "ban" or
use the robots.txt to give instructions about their site to web robots.
There are "good" bots and "bad" bots, the "good" ones are usually obediant and respect any "instructions", the "bad" ones don't.
kedaha wrote:I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.
This is a list, that is used in the phpBB software,

Code: Select all

 Google [Bot] 	 	
Yahoo [Bot] 	 	
Bing [Bot] 	  	
AdsBot [Google]  	
Alexa [Bot] 		
Alta Vista [Bot]  	
Ask Jeeves [Bot]  	
Baidu [Spider] 	 	
Exabot [Bot] 	  	
FAST Enterprise   	
FAST WebCrawler  	
Francis [Bot] 	 	
Gigabot [Bot] 	 	
Google Adsense [Bot] 	   	
Google Desktop 	  	
Google Feedfetcher 	 	
Heise IT-Markt [Crawler] 		
Heritrix [Crawler] 	 	
IBM Research [Bot] 	  	
ICCrawler - ICjobs 		
MSN NewsBlogs 	 	
MSN [Bot] 		
MSNbot Media 	  	
Majestic-12 [Bot] 	  	
Metager [Bot] 	 	
NG-Search [Bot] 		
Nutch [Bot] 		
Nutch/CVS [Bot] 	 	
OmniExplorer [Bot] 	 	
Online link [Validator] 	 	
SEO Crawler 	  	
SEOSearch [Crawler] 	  	
Seekport [Bot] 	 	
Sensis [Crawler] 	  	
Seoma [Crawler] 		
Snappy [Bot] 	 	
Steeler [Crawler] 	   	
Synoo [Bot] 	  	
Telekom [Bot] 	 	
TurnitinBot [Bot] 	 	
Voyager [Bot] 	  	
W3 [Sitesearch] 	  	
W3C [Linkcheck] 	  	
W3C [Validator] 		
WiseNut [Bot] 	 	
Yacy [Bot] 		
Yahoo MMCrawler [Bot] 	
Yahoo Slurp [Bot] 	 	
YahooSeeker [Bot] 	 	
ichiro [Crawler] 	   	
psbot [Picsearch] 
They can be edited, deleted , etc :
For example:

Code: Select all

Bot name 	Last visit 	Options 	Mark
Google [Bot] 	 Mon Oct 31, 2016 4:58 pm  	 Deactivate  	 Edit  	 Delete 
I got this from a different forum, one that I administer, On this forum I am not
a admin, and don't know , nor have access to how it is set up.
Post by roseway » but it baffles me why they need to be manually registered.
I am sorry, maybe I worded previous post poorly, they are not "manually" registered,
it is more of a "default" list, and it they are not listed as "registered" users, it is a seperate group. " Spiders/Robots ",
Manage bots
“Bots”, “spiders” or “crawlers” are automated agents most commonly used by search engines to update their databases. Since they rarely make proper use of sessions they can distort visitor counts, increase load and sometimes fail to index sites correctly. Here you can define a special type of user to overcome these problems.
And that is where it gets to be a "manual" process, when the admins need to "define"
special types of users, etc. Decide which bots to allow, and where they are allowed
to go, etc,......

There are some really "bad bots", that even go so far as to try to "look like "
the google bot, (for example), I think these are what give the "good bots" such a
bad reputation,
by Bulkley » I say should because I'm never quite sure that's all they do.
I think in many cases, and this happened to me, I thought it was a "google bot",
because that was what it said it was, but it was trying to change it's "profile" as
if it was a normal user, so my imediate reaction was to "ban" and block the google
bot, but later when I was asking about it, on another forum, I learned that it was not
a real google bot, the real google bot does not try to do that sort of thing.
Adminstrating a server/website, forums, etc. is much more complicated then
many people realize, it is important that the owner keeps track of what the "bots"
and visisitors are doing, if the site owner can not, or does not want to , then they
need to have administrators that can be trusted, and know how to deal with those
"tasks",... ..
I don't really know that much, in fact I just barely "scratch" the surface, all though
it is something that interests me, and I am constantly trying to learn more,....

User avatar
4D696B65
Site admin
Site admin
Posts: 2696
Joined: 2009-06-28 06:09
Been thanked: 85 times

Re: Bing [Bot]

#7 Post by 4D696B65 »

kedaha wrote:I spotted another one today: Majestic-12 [Bot]; I wonder how many there are.
We have 51 registered. Most log in as invisible.

millpond
Posts: 698
Joined: 2014-06-25 04:56

Re: Bing [Bot]

#8 Post by millpond »

Search engine bots are quite common on all public sites - and some sites go out of their way to invite them in and customize listings for them.

Myself, i block them at robots.txt and .htaccess, which of course means that searching for my site will come up with zero results.
The way I want it.

There are logical reasons why this site would want exposure to search engines, and it wouldnt make a difference how diabolocal they wrere. M$ is hardly, at heart, any worse than google.
All out to get control of your info and your desktop.

tomazzi
Posts: 730
Joined: 2013-08-02 21:33

Re: Bing [Bot]

#9 Post by tomazzi »

OK, so we have 51 "spies" - but this is not a reply for my original question: (re-formed):
So, what is the BingBot doing here - listed as a registered user?

If the code of this bot requires listing as a "normal" user - then it simply means a bug in the code - and if this is true (personally I doubt it) - then such bot should be simply kicked-out (as a buggy software)

Another thing is, that Google, whether You like it or not (I don't), has ~75-80% of the search engine market share, so who cares about Bing?

... Especially that Bing was proven to steal search results from Google... (I'm prepared to provide a link - max few seconds of searching)

... And additionally, the company which stays behind the Bing (MSFT) is a long-time declared foe of Debian or Linux in general -> i.e. this is a conflict of interests.

Regards.
Odi profanum vulgus

tomazzi
Posts: 730
Joined: 2013-08-02 21:33

Re: Bing [Bot]

#10 Post by tomazzi »

No reply is a reply - it means that debian forums is not an independant organization - it actually shows that the debian *is* dependant on external companies - and all the debian project declarations regarding the care of user's experience/rights are worth crap. No surprise here - the way in which the adoption of systemd made by RedHat was executed was just a tip of the iceberg.

Now I've learned something: Debian is just a puppet - it does not have any targets, especially after Ian have
*accidencially* died - and the users of the famous, stable "debian" are now on their own.

No reply is a reply - sometimes it's even better than having a "official" reply from a clueless puppet...

Anyway, regards.

Edit:
If the above is not clear, I'm just asking to be banned - literally and practically - please delete my account on this forums...
I know that You can't do this (freedom?) - You can only rename my account to something like "xxx-account-deleted" - so please do so...

Again, Regards...
Odi profanum vulgus

User avatar
GarryRicketson
Posts: 5644
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Bing [Bot]

#11 Post by GarryRicketson »

The question did get answered, the bing bot, is doing the same thing all
the other search engine bots are doing, getting data to add to the search
engine,.........
But in any event, the proper proceedure to get a user account removed,
or with any other account problems, is explained here :

HOWTO contact forum moderators/admins
If you have problems with accounts or with posting, or having any other technical problem or question, please contact admin@forums.debian.net. The same repeat-guideline as for team@ exists here.
Thank you,
from: "from a clueless puppet"
Last edited by GarryRicketson on 2016-11-24 23:01, edited 1 time in total.

User avatar
Head_on_a_Stick
Posts: 14114
Joined: 2014-06-01 17:46
Location: London, England
Has thanked: 81 times
Been thanked: 132 times

Re: Bing [Bot]

#12 Post by Head_on_a_Stick »

@tomazzi, what does the presence of a bot on these boards have to do with debian.org?

AFAIUI, these forums are independent and not officially part of the Debian project.
deadbang

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: Bing [Bot]

#13 Post by dasein »

tomazzi wrote:... who cares about Bing?
According to your own "estimate," one searcher in every five.

IMO, banning a(ny) legitimate search engine spider would violate DFSG.
tomazzi wrote: please delete my account on this forums...
Absolutely no need for melodrama. If you're determined to leave, simply change your password blindly (not looking at the screen or keyboard) to a bunch of random keypresses. You'll be just as gone, without imposing a pointless and unnecessary burden on the staff.

Post Reply