cat-fanciers.com
robots.txt

Robots Exclusion Standard data for cat-fanciers.com

Resource Scan

Scan Details

Site Domain cat-fanciers.com
Base Domain cat-fanciers.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-02-06T12:21:17+00:00
Next Scan 2025-04-07T12:21:17+00:00

Last Successful Scan

Scanned2024-11-16T08:32:50+00:00
URL https://cat-fanciers.com/robots.txt
Redirect https://www.cat-fanciers.com/robots.txt
Redirect Domain www.cat-fanciers.com
Redirect Base cat-fanciers.com
Domain IPs 34.213.124.12
Redirect IPs 34.213.124.12
Response IP 34.213.124.12
Found Yes
Hash ef169deea38fb770dc776cf711154cdc50635af04ec60122ec4d72541b4c53b1
SimHash a10c5a607575

Groups

*

Rule Path
Allow /order/track
Disallow /order/
Disallow /account/
Disallow /manager/
Disallow /*/filter/
Disallow /*/product/
Disallow /*/color/
Disallow /*/department/
Disallow /*/style/
Disallow /contact/email

googlebot

Rule Path
Disallow /orders/receipt

teegraspbot

Rule Path
Disallow /

teeraidbot

Rule Path
Disallow /

facebot/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebot/1.1

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.cat-fanciers.com/sitemap-index.xml