fatquartershop.com
robots.txt

Robots Exclusion Standard data for fatquartershop.com

Resource Scan

Scan Details

Site Domain fatquartershop.com
Base Domain fatquartershop.com
Scan Status Ok
Last Scan2024-09-17T05:14:29+00:00
Next Scan 2024-10-17T05:14:29+00:00

Last Scan

Scanned2024-09-17T05:14:29+00:00
URL https://fatquartershop.com/robots.txt
Redirect https://www.fatquartershop.com/robots.txt
Redirect Domain www.fatquartershop.com
Redirect Base fatquartershop.com
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 151.101.129.124
Found Yes
Hash b81d0759d00d5a721f2400bc96e975d965ab0040281b2ac55cf15431efe36242
SimHash 3824f0914631

Groups

*
screaming frog seo spider

Rule Path
Allow /*?p=
Allow /media/
Disallow */account/
Disallow */address/
Disallow */app/
Disallow */bin/
Disallow */cart/
Disallow */catalogsearch/
Disallow */checkout/
Disallow */contacts/
Disallow */control/
Disallow */customer/
Disallow */CVS/
Disallow */dev/
Disallow */giftcard/
Disallow */guest/
Disallow */index.php/
Disallow */newsletter/
Disallow */order/
Disallow */phpserver/
Disallow */product_compare/
Disallow */review/
Disallow */sales/
Disallow */sendfriend/
Disallow */storecredit/
Disallow */setup/
Disallow */update/
Disallow */vault/
Disallow */vendor/
Disallow */wishlist/
Disallow /catalog/product/view/
Disallow /review/product/list/id/
Disallow /error_log
Disallow */404/
Disallow */cgi-bin/
Disallow */downloader/
Disallow */errors/
Disallow */includes/
Disallow */lib/
Disallow */pkginfo/
Disallow */report/
Disallow */shell/
Disallow */var/
Disallow */christmascloth/
Disallow */itssewemma/
Disallow */ecc/
Disallow */magmi/
Disallow */scripts/
Disallow */.well-known/pki-validation/
Disallow /poll/
Disallow /tag/
Disallow /catalogsearch/result/
Disallow */CHANGELOG.md
Disallow */composer.json
Disallow */composer.lock
Disallow */CONTRIBUTING.md
Disallow */CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow */COPYING.txt
Disallow */Gruntfile.js
Disallow */ISSUE_TEMPLATE.md
Disallow */LICENSE.html
Disallow */LICENSE.txt
Disallow */LICENSE_AFL.txt
Disallow */nginx.conf.sample
Disallow */package.json
Disallow */package.json.sample
Disallow */php.ini.sample
Disallow */RELEASE_NOTES.txt
Disallow /cron.php
Disallow /cron.sh
Disallow /install.php
Disallow /LICENSE_EE.html
Disallow /LICENSE_EE.txt
Disallow /*.php$
Disallow /*SID%3D
Disallow /*.CVS
Disallow /*.Idea$
Disallow /*.Sql$
Disallow /*.Svn$
Disallow /*.Tgz$
Disallow /*.Zip$
Disallow /*?*=

storebot-google
mediapartners-google
adsbot-google

Rule Path
Allow /

baiduspider
yisouspider
petalbot
ahrefsbot
dotbot
mj12bot
rogerbot
brandverity

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

yahoo-blogs/v3.9

Rule Path
Disallow /

yandex

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

brands-bot

Rule Path
Disallow /

brands-bot-logo

Rule Path
Disallow /

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.fatquartershop.com/sitemap.xml

Comments

  • Allow List
  • Disallow Directories
  • Disallow Old Directories
  • Disallow Files & Parameters
  • Google's Ad & Shopping Bots
  • Disallow Bad Bots

Warnings

  • 1 invalid line.