in Meddling, MT.Net, X-Geek

MSN now snooping anonymously

In a very strange occurrence, my website got visited from what appears to be an MSN spider that didn’t identify itself (fake user agent has been highlighted below):

65.55.231.117 – – [22/Oct/2009:10:02:07 -0400] “GET /robots.txt HTTP/1.1” 200 24 “-” “Mozilla/4.0”
65.55.231.117 – – [22/Oct/2009:10:02:07 -0400] “GET /wp-content/uploads/2009/10/oculan-screenshot-300×230.png HTTP/1.1” 200 120896 “-” “Mozilla/4.0”
65.55.210.80 – – [22/Oct/2009:10:02:20 -0400] “GET /page/2/?q=node%2F1699 HTTP/1.1” 200 29922 “-” “msnbot/1.1 (+http://search.msn.com/msnbot.htm)”
65.55.230.228 – – [22/Oct/2009:10:08:13 -0400] “GET /robots.txt HTTP/1.1” 200 24 “-” “Mozilla/4.0”
65.55.230.228 – – [22/Oct/2009:10:08:13 -0400] “GET /2009/10/15/big-names-in-sources-of-suspicious-traffic/ HTTP/1.1” 200 10502 “-” “Mozilla/4.0”

65.55.230.228 resolves to msnbot-65-55-230-228.search.msn.com. 65.55.231.117 is a Microsoft address but doesn’t have an entry in DNS.

Just to make sure someone wasn’t spoofing the MSN namespace, I checked the whois record for these host. Sure enough, they belong to Microsoft:

OrgName: Microsoft Corp
OrgID: MSFT
Address: One Microsoft Way
City: Redmond
StateProv: WA
PostalCode: 98052
Country: US

NetRange: 65.52.0.0 – 65.55.255.255
CIDR: 65.52.0.0/14
NetName: MICROSOFT-1BLK
NetHandle: NET-65-52-0-0-1
Parent: NET-65-0-0-0-0
NetType: Direct Assignment
NameServer: NS1.MSFT.NET
NameServer: NS5.MSFT.NET
NameServer: NS2.MSFT.NET
NameServer: NS3.MSFT.NET
NameServer: NS4.MSFT.NET
Comment:
RegDate: 2001-02-14
Updated: 2004-12-09

RTechHandle: ZM23-ARIN
RTechName: Microsoft Corporation
RTechPhone: +1-425-882-8080
RTechEmail: noc@microsoft.com

OrgAbuseHandle: ABUSE231-ARIN
OrgAbuseName: Abuse
OrgAbusePhone: +1-425-882-8080

I’m used to MSN spidering my site, but it’s always identified itself before. Why did it not identify itself?


Update 3:50 PM ET:
Apparently I’m not the first to discover this unusual traffic.