Search Engine Spiders which may visit your site
209.67.247.156 - Fast WebCrawler/2.1 pre14
209.185.143.98 - Mozilla/3.0 (Slurp/s slurp@inktomi.com
216.35.103.42 - Slurp.so/1.0 slurp@inktomi.com
209.185.108.147 - Googlebot/1.0 googlebot@googlebot.com
209.67.229.101 - bos-spider10b.bos.lycos.com Lycos_Spider_(T-Rex)
204.162.96.124 - InfoSeek Sidewinder/0.9
198.3.103.97 - Excite ArchitextSpider
208.219.77.19 - NorthernLight Gulliver/1.3
There are 100's of spiders which roam the web many of which
are brothers and sisters of the above list
You should always have a good web statistics available for
your site to see where the visitors to your site and the
spiders came from.
You may also receive visits from not so friendly spiders
like email harvester spiders which are only visiting your
site to retrieve email addresses, or unknown spiders which
are roaming the Internet gathering data for what ever reason
More Examples of Spiders and Clients which may visit
your site
Openfind data gatherer@Openbot
flunky crawler_admin@bigfoot.com
parallelContextFocusCrawler1.1
larbin_2.2.0 (crawl@compete.com)
Bot mailto:craftbot@yahoo.com
SpaceBison
webbandit
BunnySlippers
ScoutAbout
Ziggy The Clown From Hell!!
LinkWalker
LexiBot
BaiDuSpider
DigOut4U
KIT Fireball
Microsoft URL Control 5.01.4319
Microsoft URL Control 6.00.8862
moget
Xenu_s Link Sleuth 1.0r
URL Indexer
sitecheck.internetseer.com
MFC_Tear_Sample
Nokia7110
Nutscrape
TV33_Mercator_1 1.0
teomaagent crawler admin@teoma.com
bumblebee@relevare.com
Mitsu
CSE@IITBombay
MARS SV
AtlantisSearch
RepoMonkey Bait & Tackle
spider.yellopet.com www.yellopet.com
The Intraformant
SwishSpider
SlySearch (slysearch@slysearch.com)
AmigaVoyager
Toutatis 2.5 2 (http:
GaisLab data gatherer, Gaisbot
Zeus ThemeSite Viewer Webster Pro V2.9 Win32
WIBBLE WOBBLE
Surfnomore Spider v1.1
WebSauger 1.20b
UIowaCrawler
Kenjin Spider
Robot@SuperSnooper.Com
16274.345.67.23 WebWasher 3.0
NOSEYBOLLOCKS V1.0
GB2 LinkChecker
EmailWolf 1.00
EmailSiphon
And many many more
