pecascarlos.es
robots.txt

Robots Exclusion Standard data for pecascarlos.es

Resource Scan

Scan Details

Site Domain pecascarlos.es
Base Domain pecascarlos.es
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-24T00:38:40+00:00
Next Scan 2026-03-24T00:38:40+00:00

Last Successful Scan

Scanned2025-05-28T00:39:05+00:00
URL https://pecascarlos.es/robots.txt
Redirect https://www.pecascarlos.es/robots.txt
Redirect Domain www.pecascarlos.es
Redirect Base pecascarlos.es
Domain IPs 104.21.50.68, 172.67.158.116, 2606:4700:3032::ac43:9e74, 2606:4700:3033::6815:3244
Redirect IPs 104.21.50.68, 172.67.158.116, 2606:4700:3032::ac43:9e74, 2606:4700:3033::6815:3244
Response IP 172.67.158.116
Found Yes
Hash 43afff64c6a573ba579192a7614a6eb720aa316576155d08c0a2e49593985bab
SimHash 29169861f1d0

Groups

*

Rule Path
Disallow /admin

adsbot-google

Rule Path
Disallow /admin

nutch

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap /sitemaps.xml

Comments

  • Google adsbot ignores robots.txt unless specifically named!