Forum Topic: How to block Httrack Website Copier ? (its options)

Forum: .htaccess Forum : Security • Posted by Eduardo Quevedo • Updated:

Hi,

Recently someone has cloned my website using Httrack Website Copier.

After that, my website diplomadosidg.com has .htaccess with “2013 User Agent Blacklist” and “5G Blacklist 2013” (from perishablepress.com), and robots.txt with

User-agent: Httrack Disallow /

I downloaded HTTrack (it’s free). It (.htaccess) works whit HTTrack, but it fails (don’t stop the copy process) when Httrack use its options “no robots.txt rules” and “Browser Identity: none”.

Thanks.

1 Reply to “How to block Httrack Website Copier ? (its …”

Posted by Jeff Starr •

Any site that is accessible on the web can be copied. In order to stop someone/something from copying it automatically (e.g., via httrack) the requests need to have something in common. Best case scenario the User Agent can be blocked. If the UA is not specified or blank, then it is possible to block that, but doing so is not recommended because some legitimate requests are known to use empty/blank UAs as well.