Hmm.. it depends on the host and server configuration, but the server response (whatever it is) should be included with each log entry. For example, most requests should be 200 “Ok”, but if you search for other responses “403” and “500” for example, they should turn up (assuming they are there); otherwise, I would ask your host the same question posted above just in case they’ve done something or know of an easy solution.

Beyond asking your host, check for a server error log, which is generally separate than the server access log. The error log includes more specific information about errors, such as those that trigger, say, a 403 response.

Beyond that, it gets even more complex as you go.. you also need to keep an eye out for the *type* of requests being made. For example, most requests are of the GET variety, but there are also scripts that do nothing all day but make POST requests to hundreds of thousands of sites. POST requests also should be included in the access logs. The trick, however, is that the actual POST content is not included in the logged request. So all of that to say this, it’s possible to write PHP scripts that will record all/any server/request info you like. For example, to cover the POST situation while also recording other types of errors, 403, 500, et al, read thru this article:

https://perishablepress.com/protect-post-requests/

Specifically, in the section “POST Control & Protection” there is a quick PHP/.htaccess technique that may be modified to log just about anything.

Well there’s no 403’s, which is odd since I’ve been knee-deep in 6G for the past few days, and triggering 403’s every six seconds. :)

OK, so I emailed the server guy, we’ll see what he says; thanks.

OK now I know why I’m not seeing any 403 errors, it’s because I configured htaccess to use custom error pages, so any time a 403 forbidden is triggered it’s returning a 302 found — because it found the custom error 403 page. LOL.

Jesus Christ this stuff man, it never ends. :)

So there must be a simple way to keep an eye on 403 errors, like a php snippet inserted into my 403 page that keeps track of what IP ends up there, and why.

Ah that explains it — and here is that snippet for logging such requests:

https://perishablepress.com/protect-post-requests/

Specifically, in the section “POST Control & Protection” there is a quick PHP/.htaccess technique that may be modified to log just about anything of interest.

I ended up grabbing some of the PHP code from your blackhole script, so now it sends an email to my webmaster account whenever the custom 403, etc., pages are accessed, and it includes the referer, which isn’t perfect but it’s the best solution I could figure out.