Log In Sign Up

fairingworlds.com
robots.txt

Robots Exclusion Standard data for fairingworlds.com

Resource Scan

Scan Details

Site Domain fairingworlds.com
Base Domain fairingworlds.com
Scan Status Ok
Last Scan2025-09-28T14:04:42+00:00
Next Scan 2025-10-28T14:04:42+00:00

Last Scan

Scanned2025-09-28T14:04:42+00:00
URL https://fairingworlds.com/robots.txt
Domain IPs 193.203.179.159
Response IP 193.203.179.159
Found Yes
Hash e5530fd0b69420098cfe1733b7d0152e3b5a95421d6e6d0fedec73568284ab63
SimHash 726197fec239

Groups

*

Rule Path
Allow /
Disallow /.well-known/amphtml/apikey.pub

Other Records

Field Value
crawl-delay 20

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

*

Rule Path
Disallow /?sort=
Disallow /?limit=
Disallow /index.php?route=product%2Fcatalog
Disallow /index.php?route=product%2Fcompare
Disallow /index.php?route=account%2Freward
Disallow /index.php?route=information%2Fcontact
Disallow /index.php?route=account%2Freturn%2Fadd
Disallow /index.php?route=information%2Fcontact
Disallow /index.php?route=account%2Fwishlist
Disallow /privacy
Disallow /terms
Back to top

Warnings

  • 1 invalid line.
Back to top