[Hampshire] Strangeness in apache logs: many HEAD requests

Top Page

Reply to this message
Author: Victor Churchill
Date:  
To: Hampshire LUG Discussion List
Subject: [Hampshire] Strangeness in apache logs: many HEAD requests
I am seeing lots of occurences of the following kind of thing:
1. POST request running a script on the server
2. GET request for an image generated in response to (1)
3. many HEAD requests for other images generated in the past but
unrelated to this invocation of (1).

The HEADs come from different IP addresses all in the same /24 block
as the originating request; lots of them in the space of a few
seconds.

The original request comes from someone who looks like they are using
an AOL-provided browser.

There may be as many as 60+ of the HEAD requests for each GET.

Example (sanitized):

aaa.bbb.ccc.67 200 1235535 [03/May/2007:18:40:47 +0100] "POST
/script?CGISESSID=407558d0647970ef18b970d9f86e2332 HTTP/1.1"
"Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; SV1; .NET
CLR 1.1.4322)"
aaa.bbb.ccc.70 200 4306 [03/May/2007:18:41:01 +0100] "GET img1.png
HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT
5.1; SV1; .NET CLR 1.1.4322)"
aaa.bbb.ccc.37 200 - [03/May/2007:18:41:36 +0100] "HEAD /img2.png HTTP/1.1" "-"
aaa.bbb.ccc.38 200 - [03/May/2007:18:41:36 +0100] "HEAD /img3.png HTTP/1.1" "-"
aaa.bbb.ccc.101 200 - [03/May/2007:18:41:36 +0100] "HEAD /img4.png
HTTP/1.1" "-"
aaa.bbb.ccc.132 200 - [03/May/2007:18:41:36 +0100] "HEAD /img5.png
HTTP/1.1" "-"
aaa.bbb.ccc.129 200 - [03/May/2007:18:41:36 +0100] "HEAD /img6.png
HTTP/1.1" "-"
aaa.bbb.ccc.3 200 - [03/May/2007:18:41:36 +0100] "HEAD /img7.png HTTP/1.1" "-"
aaa.bbb.ccc.70 200 - [03/May/2007:18:41:36 +0100] "HEAD /img8.png HTTP/1.1" "-"
aaa.bbb.ccc.67 200 1228026 [03/May/2007:18:41:36 +0100] "POST
/script?CGISESSID=407558d0647970ef18b970d9f86e2332 HTTP/1.1"
"Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; SV1; .NET
CLR 1.1.4322)"
aaa.bbb.ccc.71 200 4054 [03/May/2007:18:41:49 +0100] "GET /img9.png
HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT
5.1; SV1; .NET CLR 1.1.4322)"
aaa.bbb.ccc.129 200 - [03/May/2007:18:42:54 +0100] "HEAD /img6.png
HTTP/1.1" "-"
aaa.bbb.ccc.3 200 - [03/May/2007:18:42:54 +0100] "HEAD /img7.png HTTP/1.1" "-"
aaa.bbb.ccc.132 200 - [03/May/2007:18:42:54 +0100] "HEAD /img5.png
HTTP/1.1" "-"
aaa.bbb.ccc.38 200 - [03/May/2007:18:42:54 +0100] "HEAD /img3.png HTTP/1.1" "-"
aaa.bbb.ccc.3 200 - [03/May/2007:18:42:54 +0100] "HEAD /img10.png HTTP/1.1" "-"
aaa.bbb.ccc.7 200 - [03/May/2007:18:42:54 +0100] "HEAD /img11.png HTTP/1.1" "-"
aaa.bbb.ccc.101 200 - [03/May/2007:18:42:54 +0100] "HEAD /img4.png
HTTP/1.1" "-"
aaa.bbb.ccc.37 200 - [03/May/2007:18:42:54 +0100] "HEAD /img12.png
HTTP/1.1" "-"
aaa.bbb.ccc.70 200 - [03/May/2007:18:42:54 +0100] "HEAD /img8.png HTTP/1.1" "-"
aaa.bbb.ccc.67 200 1234508 [03/May/2007:18:42:54 +0100] "POST
/script?CGISESSID=407558d0647970ef18b970d9f86e2332 HTTP/1.1"
"Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT 5.1; SV1; .NET
CLR 1.1.4322)"
aaa.bbb.ccc.9 200 5232 [03/May/2007:18:43:09 +0100] "GET /img13.png
HTTP/1.1" "Mozilla/4.0 (compatible; MSIE 6.0; AOL 9.0; Windows NT
5.1; SV1; .NET CLR 1.1.4322)"
aaa.bbb.ccc.129 200 - [03/May/2007:18:43:44 +0100] "HEAD /img6.png
HTTP/1.1" "-"
aaa.bbb.ccc.132 200 - [03/May/2007:18:43:44 +0100] "HEAD /img5.png
HTTP/1.1" "-"
aaa.bbb.ccc.37 200 - [03/May/2007:18:43:44 +0100] "HEAD /img12.png
HTTP/1.1" "-"
aaa.bbb.ccc.3 200 - [03/May/2007:18:43:44 +0100] "HEAD /img7.png HTTP/1.1" "-"
aaa.bbb.ccc.38 200 - [03/May/2007:18:43:44 +0100] "HEAD /img3.png HTTP/1.1" "-"
aaa.bbb.ccc.7 200 - [03/May/2007:18:43:44 +0100] "HEAD /img11.png HTTP/1.1" "-"
aaa.bbb.ccc.101 200 - [03/May/2007:18:43:44 +0100] "HEAD /img4.png
HTTP/1.1" "-"
aaa.bbb.ccc.98 200 - [03/May/2007:18:43:44 +0100] "HEAD /img14.png
HTTP/1.1" "-"
aaa.bbb.ccc.3 200 - [03/May/2007:18:43:44 +0100] "HEAD /img10.png HTTP/1.1" "-"
aaa.bbb.ccc.70 200 - [03/May/2007:18:43:44 +0100] "HEAD /img8.png HTTP/1.1" "-"

I assume that :
1.this user has AOL as their ISP and their requests come through AOL
proxy servers (nslookup aaa.bbb.ccc.ddd confirm this)
2. AOL is kindly doing a load of HEAD reuests just in case the user
might be about to ask it to fetch some image that it has cached from a
previous query in the past.

Questions:
1. How much impact does this quantity of HEAD requests have on my
server? (Is there a way I can find out?)
2. Should I be concerned, and is there anything I can/could/should do
about it anyway? (possibly even just screening them out of my logs..)

cheers
victor