poetryfashion.co.uk
robots.txt

Robots Exclusion Standard data for poetryfashion.co.uk

Resource Scan

Scan Details

Site Domain poetryfashion.co.uk
Base Domain poetryfashion.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-17T02:07:43+00:00
Next Scan 2024-06-16T02:07:43+00:00

Last Successful Scan

Scanned2024-03-26T02:06:16+00:00
URL https://poetryfashion.co.uk/robots.txt
Redirect https://www.poetryfashion.co.uk/robots.txt
Redirect Domain www.poetryfashion.co.uk
Redirect Base poetryfashion.co.uk
Domain IPs 104.26.6.246, 104.26.7.246, 172.67.75.30, 2606:4700:20::681a:6f6, 2606:4700:20::681a:7f6, 2606:4700:20::ac43:4b1e
Redirect IPs 104.26.6.246, 104.26.7.246, 172.67.75.30, 2606:4700:20::681a:6f6, 2606:4700:20::681a:7f6, 2606:4700:20::ac43:4b1e
Response IP 104.26.7.246
Found Yes
Hash 93bbc62367b81326818587fd55e512d0954495c73d2775d4324454cf438b37ed
SimHash 6616dc71e6e1

Groups

*

Rule Path
Disallow /email-newsletter.htm
Disallow /request-catalogue.htm
Disallow /vieword.csp
Disallow /view.csp
Disallow /save.csp
Disallow /emailfriend.csp
Disallow /emailfriend/
Disallow /ajax*
Disallow /csp/secure/
Disallow /csp/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

nutch

Rule Path
Disallow /