Log In Sign Up

forum-des-portables-asus.fr
robots.txt

Robots Exclusion Standard data for forum-des-portables-asus.fr

Resource Scan

Scan Details

Site Domain forum-des-portables-asus.fr
Base Domain forum-des-portables-asus.fr
Scan Status Ok
Last Scan2024-07-05T15:44:03+00:00
Next Scan 2024-07-12T15:44:03+00:00

Last Scan

Scanned2024-07-05T15:44:03+00:00
URL https://forum-des-portables-asus.fr/robots.txt
Domain IPs 167.114.164.166
Response IP 167.114.164.166
Found Yes
Hash 9e6e36065f64dce301594897fbe64a862a13963051196f977908899b48725522
SimHash 58742b1b585b

Groups

*

Rule Path
Disallow /admin/
Disallow /blackhole/
Disallow /forums/BIOS/
Disallow /asus/
Disallow /backup/
Disallow /isos/
Disallow /js/
Disallow /forums/wiki/
Disallow /wiki
Disallow /www/
Disallow /XF2_Addons/
Disallow /forums/find-new/
Disallow /forums/account/
Disallow /forums/goto/
Disallow /forums/login/
Disallow /forums/admin.php
Disallow /forums/conversations/
Disallow /forums/devforums/
Disallow /forums/rating/
Allow /
Allow /forums/

adscanner

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

infohelfer

Rule Path
Disallow /

pixray*

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

u

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

nutch-1.4

Rule Path
Disallow /

discobot

Rule Path
Disallow /

spiderlytics

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

alexa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

macinroy

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

domaintuno

Rule Path
Disallow /

dubaiindex

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

abonti

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

ncbot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

it2media-domain-crawler

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

psbot

Rule Path
Disallow /

woriobot

Rule Path
Disallow /

ssearch

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

waybackarchive.org

Rule Path
Disallow /

netestate

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

obot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

exabot

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

admantx-adform/3.1

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.forum-des-portables-asus.fr/forums/sitemap.php
Back to top

Comments

  • metadatalabs.com
  • Ahrefs.com (http://ahrefs.com/robot/)
  • IP 5.10.83.36
  • "Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)"
  • ezooms.bot
  • domaintools.com
  • www.infohelfer.de
  • www.pixray.com
  • warebay.com
  • aihit.com
  • yandex.com YandexBot YandexImages
  • IP 141.8.147.17
  • "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)"
  • U
  • unister.de
  • www.Nutch.de
  • IP 62.146.2.234, 117.78.13.18
  • "Domnutch-Bot/Nutch-1.0 (Domnutch; http://www.Nutch.de/)"
  • SEO Spider spider@spiderlytics.com
  • IP 5.199.136.130
  • "Mozilla/5.0 (compatible; Spiderlytics/1.0; +spider@spiderlytics.com)"
  • Unknown
  • IP 207.241.226.239
  • "ia_archiver(OS-Wayback)"
  • crawler@alexa.com
  • IP 204.236.235.245
  • "ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com)"
  • Unknown
  • IP 108.59.8.70
  • "Mozilla/5.0 (compatible; MJ12bot/v1.4.4; http://www.majestic12.co.uk/bot.php?+)"
  • http://go.mail.ru/help/robots
  • IP 217.69.133.253
  • "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)"
  • macinroy.com
  • IP 85.25.137.24
  • "MacInroy Privacy Auditors. See jarnold.org's privacy violation report: http://jarnold.org.macinroy.com/jarnold.org"
  • www.semrush.com/bot.html
  • IP 46.229.164.102
  • "Mozilla/5.0 (compatible; SemrushBot/0.97; +http://www.semrush.com/bot.html)"
  • http://www.icjobs.de
  • IP 85.25.71.40
  • "Mozilla/5.0 (X11; U; Linux i686; de; rv:1.9.0.1; compatible; iCjobs Stellenangebote Jobs; http://www.icjobs.de) Gecko/20100401 iCjobs/3.2.3"
  • http://fulltext.sblog.cz
  • IP 77.75.77.32
  • "SeznamBot/3.0 (+http://fulltext.sblog.cz/)"
  • http://webmeup-crawler.com
  • IP 108.178.53.146
  • "Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup-crawler.com/)"
  • http://siteexplorer.info
  • IP 208.43.225.84
  • "Mozilla/5.0 (compatible; SiteExplorer/1.0b; +http://siteexplorer.info/)"
  • www.linkdex.com/about/bots
  • IP 54.242.123.170, 23.22.229.75, 54.225.52.217 23.20.126.233
  • "Mozilla/5.0 (compatible; linkdexbot/2.0; +http://www.linkdex.com/about/bots/)"
  • www.wotbox.com/bot
  • IP 81.144.138.34
  • "Wotbox/2.01 (+http://www.wotbox.com/bot/)"
  • http://www.domaintuno.com
  • IP 192.96.204.42
  • "http://www.domaintuno.com/whois/jarnold.org" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)"
  • unknown addressendeutschland.de
  • IP 86.109.249.174
  • "http://arnold-soft.de/" "dubaiindex (addressendeutschland.de)"
  • www.pagesinvenotry.com
  • IP 130.185.109.243
  • "PagesInventory (robot http://www.pagesinvenotry.com)"
  • www.abonti.com
  • IP 77.233.225.115
  • "Mozilla/5.0 (compatible; Abonti/0.91 - http://www.abonti.com)"
  • www.backlinktest.com/crawler.html
  • IP 46.4.100.231
  • "BacklinkCrawler (http://www.backlinktest.com/crawler.html)"
  • http://netcomber.com
  • IP 54.227.175.17
  • "NCBot http://netcomber.com?st=ba2Tool for finding all their domain names."
  • Unknown
  • IP 69.58.178.58
  • "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:14.0; ips-agent) Gecko/20100101 Firefox/14.0.1"
  • www.grapeshot.co.uk/crawler.php
  • IP 89.145.95.2
  • "Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)"
  • www.80legs.com/webcrawler.html
  • IP 64.125.222.16
  • "Mozilla/5.0 (compatible; 008/0.83; http://www.80legs.com/webcrawler.html;) Gecko/2008032620"
  • it2media.de
  • IP 86.109.249.169
  • "it2media-domain-crawler/1.0 on crawler-prod.it2media.de"
  • http://crawler.sistrix.net
  • IP 176.9.148.197, IP 176.9.155.226, 5.9.112.66
  • "Mozilla/5.0 (compatible; SISTRIX Crawler; http://crawler.sistrix.net/)"
  • www.picsearch.com/bot.html
  • IP 217.212.224.183
  • "psbot/0.1 (+http://www.picsearch.com/bot.html)"
  • worio.com
  • IP 107.22.250.59
  • "Mozilla/5.0 (compatible; woriobot +http://worio.com)"
  • semantissimo.de
  • IP 88.198.24.173
  • "ssearch_bot (sSearch Crawler; http://www.semantissimo.de)"
  • www.archive.org/details/archive.org_bot
  • IP 207.241.237.102 + .103 (abwechselnd!) + 207.241.226.234
  • Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)"
  • +spider@waybackarchive.org
  • IP 5.199.136.130
  • "Mozilla/5.0 (compatible; waybackarchive.org/1.0; +spider@waybackarchive.org)"
  • www.website-datenbank.de
  • IP 81.209.177.145
  • "netEstate NE Crawler (+http://www.website-datenbank.de/)"
  • www.compspy.com/spider.html
  • IP 68.47.129.55
  • "Mozilla/5.0 (compatible; CompSpyBot/1.0; +http://www.compspy.com/spider.html)"
  • www.seoprofiler.com/bot
  • IP 198.199.89.149, 162.243.203.202
  • "Mozilla/5.0 (compatible; spbot/4.1.0; +http://OpenLinkProfiler.org/bot )"
  • http://filterdb.iss.net/crawler/
  • IP 206.253.226.18
  • "Mozilla/5.0 (compatible; oBot/2.3.1; http://filterdb.iss.net/crawler/)"
  • http://www.baidu.com
  • 183.60.243.187
  • "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0"
  • http://www.exabot.com/go/robot
  • IP 178.255.215.69
  • "Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)"
  • www.tiscali.it
  • IP 217.73.208.103
  • "Mozilla/5.0 (compatible; IstellaBot/1.18.81 +http://www.tiscali.it/)"
  • www.netseer.com/crawler.html
  • IP 75.98.9.250
  • "Mozilla/5.0 (compatible; NetSeer crawler/2.0; +http://www.netseer.com/crawler.html; crawler@netseer.com)"
  • http://www.opensiteexplorer.org/dotbot, help@moz.com
  • IP 208.115.113.92
  • "Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, help@moz.com)"
  • http://www.proximic.com/info/spider.php# IP 54.211.1.18
  • "Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)"
  • http://commoncrawl.org/faq/
  • IP 54.227.12.4
  • "CCBot/2.0 (http://commoncrawl.org/faq/)"
  • IP 130.211.186.147, 146.148.35.52
  • "GET / HTTP/1.0" 200 10064 "-" "NerdyBot"
  • http://semalt.semalt.com/crawler.php
  • IP 187.79.214.121
  • "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
  • User-agent: xxx
  • Disallow: /
  • IP 69.84.207.246
  • "LSSRocketCrawler/1.0 LightspeedSystems"
  • ???
  • 50.17.21.141
  • "Cliqzbot"
Back to top

Warnings

  • 4 invalid lines.
Back to top