Log In Sign Up

fabletics.co.uk
robots.txt

Robots Exclusion Standard data for fabletics.co.uk

Resource Scan

Scan Details

Site Domain fabletics.co.uk
Base Domain fabletics.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-20T12:28:31+00:00
Next Scan 2024-12-04T12:28:31+00:00

Last Successful Scan

Scanned2023-11-13T02:32:19+00:00
URL https://fabletics.co.uk/robots.txt
Redirect https://www.fabletics.co.uk/robots.txt
Redirect Domain www.fabletics.co.uk
Redirect Base fabletics.co.uk
Domain IPs 104.18.36.148, 172.64.151.108
Redirect IPs 104.18.36.148, 172.64.151.108
Response IP 172.64.151.108
Found Yes
Hash c69b46f5500f6e0c66fb6eb9d8ee5a59a3edfa90a38ce2dfe2d6c0b5c6f310c3
SimHash ed851479d10c

Groups

googlebot

Rule Path
Disallow /dms*/
Disallow /*?altoken=*
Disallow /*?alkey=*
Disallow /*?*&altoken=
Disallow /*?*&alkey=

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

snapchatadsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top