常见搜索引擎蜘蛛名称大全
分类:网站优化 Tags: 搜索蜘蛛 网站优化 搜索引擎 评论:1 浏览:10383 最近更新时间:2014/8/29 23:03:10
百度 Baiduspider
http://www.baidu.com/search/spider.htm
据说是百度MP3搜索的蜘蛛 Baidu-Transcoder
据说是百度图片搜索的蜘蛛 Baidu-Thumbnail
百度手机搜索 baiduspider-mobile-gate
Google 谷歌
1) Googlebot:从Google的网站索引和新闻索引中抓取网页
Googlebot/2.1
2) Googlebot-Mobile:针对Google的移动索引抓取网页
3) Googlebot-Image:针对Google的图片索引抓取网页
4) Mediapartners-Google:抓取网页确定 AdSense 的内容。只有在你的网站上展示 AdSense 广告的情况下,Google才会使用此漫游器来抓取您的网站。
5) Adsbot-Google:抓取网页来衡量 AdWords 目标网页的质量。只有在你使用 Google AdWords 为你的网站做广告的情况下,Google才会使用此漫游器。
http://www.google.com/bot.html 中文版请打开以下页面
http://www.google.cn/support/webmasters/bin/topic.py?topic=8843
雅虎中国
Yahoo! Slurp China
http://misc.yahoo.com.cn/help.html
Yahoo 英文
http://www.yuzhiguo.com/437/
Yahoo! Slurp
http://help.yahoo.com/help/us/ysearch/slurp
Yahoo 搜索竞价蜘蛛
Yahoo ContentMatch Crawler/2.1
Yahoo-MMCrawler 图片
有道 YodaoBot
http://www.yodao.com/help/webmaster/spider/
搜搜 Sosospider 网页蜘蛛 / Sosoimagespider 图片蜘蛛
http://help.soso.com/webspider.htm
http://help.soso.com/soso-image-spider.htm
搜狗 sogou spider
http://www.sogou.com/docs/help/webmasters.htm#07
新浪爱问 iaskspider
微软 msnbot
http://search.msn.com/msnbot.htm 打开后跳转到以下页面
http://help.live.com/help.aspx?mkt=zh-cn&project=wl_webmasters
*MSNBot:主网页爬虫(www.live.com)
*MSNBot-Media:图片及其它媒体爬虫(images.live.com)
*MSNBot-NewsBlogs:新闻及blog爬虫(search.live.com/news)
*MSNBot-Products:产品及购物爬虫(products.live.com)
*MSNBot-Academic:学术搜索爬虫(academic.live.com)
gigablast.com----Gigabot
http://www.gigablast.com/spider.html
webalta.net----WebAlta
http://www.webalta.net/ru/about_webmaster.html
国内的一个实验室的
apex.sjtu.edu.cn
Alexa ia_archiver iaarchiver
AOL sqworm
Altavista的蜘蛛:Scooter
Lycos的蜘蛛: Lycos_Spider_(T-Rex)
Alltheweb的蜘蛛: FAST-WebCrawler/
INKTOMI的蜘蛛: Slurp
ASPSeek ASPSeek
东方网景 lanshanbot
日本的一个蜘蛛 BSpider
ask.com Ask
projecthoneypot.org Java (Often spam bot)
yandex.ru Yandex bot
Exabot.com Exabot
Netcraft
SurveyBot
larbin
http://www.majestic12.co.uk MJ12bot
http://www.cuil.com twiceler
Googlebot (Search Bot)
Google Translate (Someone Behind Proxy)
Unverified Googlebot (Search Bot)
SiteUptime (Site Helper)
Baidu Spider (Search Bot)
Yahoo! Slurp (Search Bot)
NimbuBot (Site Helper)
mon.itor.us (Site Helper)
Are My Sites Up? (Site Helper)
Host Tracker (Site Helper)
SurveyBot (Search Bot)
Facebook External Hit (Crawler)
Twingly Recon (Search Bot)
PostRank Bot (Search Bot)
Kosmix Voyager (Search Bot)
TweetMeme Bot (Crawler)
OneRiot Bot (Search Bot)
Knowmore Bot (Search Bot)
Comodo Certificates Spider (Crawler)
Jayde Crawler (Search Bot)
Netcraft Survey Bot (Search Bot)
MSN/Bing Bot (Search Bot)
XML Sitemaps Generator (Site Helper)
Dotcom Bot (Site Helper)
Pingdom Bot (Site Helper)
CatchBot (Search Bot)
Stack Rambler (Search Bot)
Yandex Bot (Search Bot)
Google Feedfetcher (Feed Fetcher)
Montastic Bot (Site Helper)
Google AdsBot (Search Bot)
Speedy Spider (Search Bot)
ExaBot (Search Bot)
Panscient Bot (Search Bot)
FindLinks (Search Bot)
Gigabot (Search Bot)
Flight Deck Bot (Crawler)
MJ12 Bot (Search Bot)
YaCy-Bot (Search Bot)
obot (Crawler)
DotBot (Crawler)
Alexa Crawler (Search Bot)
Kisiwa VSE (Crawler)
ICC-Crawler (Crawler)
Website Optimization Analyzer (Site Helper)
Daum Bot (Search Bot)
Lightspeed Crawler (Crawler)
Websense Crawler (Crawler)
Google Toolbar (Browser)
DNS Digger Explorer (Crawler)
CCBot (Crawler)
L.webis (Crawler)
Ask Crawler (Search Bot)
Trend Micro Crawler (Crawler)
Yeti (Naver Bot) (Search Bot)
Netnir Rank Check (Crawler)
spbot (Crawler)
MLBot (Crawler)
Xenu's Link Sleuth (Site Helper)
Web Link Validator (Site Helper)
Mail.Ru Bot (Search Bot)
AgHaven Bot (Search Bot)
Seexie Bot (Search Bot)
Dom2Dom (Crawler)
Setoozbot (Search Bot)
ia_archiver (Search Bot)
Corbina Bot (Crawler)
RuleSpace Bot (Crawler)
Blekko ScoutJet (Search Bot)
IBM SAI Crawler (Crawler)
OmgiliBot (Search Bot)
Bixo Labs Crawler (Crawler)
PickUpJob Bot (Search Bot)
Labhoo Bot (Search Bot)
Linkspimp (Crawler)
Google Wireless Transcoder (Someone Behind Proxy)
Envolk Bot (Search Bot)
Soso Spider (Search Bot)
DuckDuckBot (Search Bot)
PHP OpenID (Site Helper)
TurnitinBot (Crawler)
Sogou Spider (Search Bot)
008 (80legs Crawler) (Crawler)
Cyveillance Bot (Crawler)
psbot (Search Bot)
KAIST Bot (Crawler)
FlaxCrawler (Search Bot)
Snapbot (Search Bot)
LinkedIn Bot (Crawler)
Careerjet Bot (Search Bot)
Camont Spider (Crawler)
Change Detection Bot (Site Helper)
TextDigger Bot (Crawler)
Statools Bot (Crawler)
my6sense Bot (Crawler)
TwengaBot-Discover (Search Bot)
CyberPatrol Webbot (Crawler)
Incapsula Service (Site Helper)
SpokeSpider (Crawler)
Sygol Bot (Search Bot)
Seznam Bot (Search Bot)
Jaxified RSS Feeder (Feed Fetcher)
Solomono Bot (Search Bot)
Youdao Bot (Search Bot)
Hailoo Bot (Search Bot)
Google Custom Search (Search Bot)
MSR-ISRCCrawler (Crawler)
SimplePie RSS Feeder (Feed Fetcher)
Keywen Bot (Search Bot)
UrlCheck (Site Helper)
Periscope It (Site Helper)
UrlCheck (Site Helper)
Discobot (Crawler)
Webmetrics Monitoring Service (Site Helper)
FindFiles.net Bot (Search Bot)
Yahoo External Cache (Crawler)
Conduit Toolbar (Browser)
FeedBurner (Feed Fetcher)
Microsoft Web Services Client (Site Helper)
ClickTale bot (Site Helper)
Hyperspin (Site Helper)
Pinterest Bot (Crawler)
Proximic Bot (Crawler)
Google Plus Share (Crawler)
Amazon aranhabot (Site Helper)
Microsoft Lync (Site Helper)
Acoon Bot (Search Bot)
elkMonitor (Site Helper)
Covario IDS (Crawler)
Yottaa (Site Helper)
JikeSpider (Crawler)
Magpie Crawler (Crawler)
Panopta (Site Helper)
GSA Crawler (Crawler)
Disguised Google Bot (Search Bot)
Spinn3r (Search Bot)
Imperva MX Server (Crawler)
CareerBot (Crawler)
Sucuri Bot (Site Helper)
OpenIndex Bot (Search Bot)
Bountii Bot (Search Bot)
Google Partner Monitor (Site Helper)
Project Wonderful Ad/Compliancy Bot (Crawler)
Yahoo Pipes (Feed Fetcher)
Microsoft Adidxbot (Crawler)
eToro Flash Client (Feed Fetcher)
tra.cx.pider (Crawler)
eToro IOS Client (Feed Fetcher)
Radian6 Bot (Crawler)
Topsy Butterfly (Search Bot)
Grepnetstat (Site Helper)
Gnip Bot (Crawler)
Updowner bot (Crawler)
NetSeer Crawler (Crawler)
EvriNid bot (Crawler)
InternetVista Site Monitor (Site Helper)
Aihit bot (Crawler)
eToro Mobile Client (Feed Fetcher)
Yunyun Spider (Search Bot)
CodeGuard Bot (Site Helper)
VaultPress Bot (Site Helper)
YourCronTask (Site Helper)
Outbrain (Crawler)
ShopWiki Bot (Search Bot)
Server Density (Site Helper)
GlobalSign Domain Verification Agent (Site Helper)
Amazon ProductAdsBot (Crawler)
Amazon CloudFront Client (Site Helper)
Rackspace Monitoring Agent (Site Helper)
SiteTruth (Site Helper)
W3C - Markup Validator (Site Helper)
W3C - Unicorn Unified Validator (Site Helper)
W3C - Internationalization Checker (Site Helper)
W3C - Nu Markup Validation Service (Site Helper)
SpotFN iPhone App (Browser)
SpotOption iPhone App (Browser)
Google Webmasters Tools Agent (Site Helper)
W3C - CSS Validator (Site Helper)
W3C - Feed Validation (Site Helper)
W3C - Mobile OK Checker (Site Helper)
W3C - Link checker (Site Helper)
http://www.botopedia.org 蜘蛛目录
-
• 做有用的网页,做有利于网络营销的网页设计
• 搭建一个好的网站内部结构
• 个人网站流量和金钱的关系
• Site:命令的运用技巧
• 如何有效增加网站外链?
• 对个人网站推广与企业网络营销的一点偏激看法
• 如何登录搜索引擎?效果才好!
• 网络营销推广的八大技巧
• 各大搜索引擎网站收录提交入口地址
• 新整理网站站长常用SEO网站优化工具
-
• 怎么把QQ帐号显示成邮箱名?如何设置qq主显示帐号?
• 解决Windows IIS网站.svg .woff .woff2字体无法正常显示问题
• 书体坊-禚效锋开发的毛泽东电脑字体全民公测版V1.0出炉了,可以免费下载用哦
• prestashop修改底部联系方式
• .ai.psd.cdr.eps图片文件直接查看缩略图补丁免费下载
• 教你通过微信公众号找到对应二维码图片
• 做外贸!如何做好自己的外贸网站?
• 教你申请美国雅虎yahoo.com后缀邮箱
• 不错的photoshop样式免费下载,asl格式,Web2.0风格导航条与按钮效果
• 苹果电脑Mac OS系统修改DNS方法教程