fabric-direct.co.uk
robots.txt

Robots Exclusion Standard data for fabric-direct.co.uk

Resource Scan

Scan Details

Site Domain fabric-direct.co.uk
Base Domain fabric-direct.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-14T03:19:54+00:00
Next Scan 2025-09-12T03:19:54+00:00

Last Successful Scan

Scanned2025-05-09T02:20:36+00:00
URL https://fabric-direct.co.uk/robots.txt
Domain IPs 104.21.74.225, 172.67.164.13, 2606:4700:3030::6815:4ae1, 2606:4700:3033::ac43:a40d
Response IP 172.67.164.13
Found Yes
Hash 29ff061fe7ca06853897154d5abda1deb28fad82e9c79e3e79f6713779ca21d7
SimHash 4b0e54608b32

Groups

dotbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /*?utm_source=*
Disallow /*?yith_wcan=*

ccbot

Rule Path
Disallow /

baiduspider
yisouspider
petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /
Allow /wp-admin/admin-ajax.php
Disallow /*%26min_price%3D*
Disallow /*%26max_price%3D*

Other Records

Field Value
crawl-delay 100

Other Records

Field Value
sitemap https://fabric-direct.co.uk/sitemap_index.xml

Warnings

  • 1 invalid line.