Posts Tagged ‘microsoft’

Is Microsoft spamming anyone else about robots.txt blocking Bing?

June 5, 2013

So Microsoft spammed me about my robots.txt again:

from: Jyoti Bhagavatula (HCL America Inc) <[email protected]>

to: “[email protected]” <[email protected]>
date: Wed, Jun 5, 2013 at 11:11 AM

subject: Robots.txt blocking Bing crawler: http://www.seifried.org/robots.txt

Hello,

I am contacting you on behalf of the Bing Search engine (http://www.bing.com/) in regards to your robots.txt file:

http://www.seifried.org/robots.txt

Our customers have alerted us that your website was partially absent from our results and we have discovered that you are blocking our crawler, named BingBot, via a disallow directive in your robots.txt file:

User-agent: msnbot

Disallow: /

User-agent: bingbot

Disallow: /

We would be pleased if you could edit your robots.txt file to allow our crawler to fetch and index your content properly, which will in turn increase traffic to your site via our search results, by including the following section:

User-agent: Bingbot

Disallow: 

I find this pretty annoying, they don’t take the time to even look at my website and see that the contact email is pretty obviously [email protected], instead they spam my DNS WHOIS email contact. Secondly, I’ve already told them several times I don’t want to let them index my site, first via robots.txt, and secondly in email replies to the spam they send me.

Is anyone else getting this emails?