Forum Topic: Agent blocks website verification (google webmasters)

Forum: .htaccess Forum : Security • Posted by Eduardo Quevedo • Updated:

I have 2013 User Agent Blacklist in htaccess. Recently I could not verify my web site in Google Webmaster tools. Testing, I found that 2/3 UA Blacklist was the problem. I put it as a comment and it worked. What is the agent that blocks the website verification?

Thanks.

3 Replies to “Agent blocks website verification (google …”

Posted by Jeff Starr •

I’m not sure, but there are a couple of useful things that you can do:

1) Compare the code given in the book to the code given on the original post:

https://perishablepress.com/2013-user-agent-blacklist/

There were some changes made recently, and if I recall, it had to do with Google.

2) Do a quick diagnosis of section 2/3 by using the halving method:

https://perishablepress.com/the-halving-method-of-identifying-problematic-code/

Once you understand how that technique works, it should enable you to locate the problem string in a few minutes.

Let me know how it goes!

Posted by Eduardo Quevedo •

Ok, I’ll try it whit the halving method.

But, there is another problem. I see crawl errors in google webmaster report (Google say: “Errors that only ocurred when your site was crawled by Googlebot-mobile for feature phones). Errors didn’t appear for desktop and appear from Oct 28 when I modified htacces (with UA blacklist) and robots.txt

Posted by Jeff Starr •

That could mean that a different bot was used and was denied access, or that there is an issue with the content or other aspect of those pages. I would maybe remove the UA Blacklist until you can sort out which agents are not able to access which content and why.