Log In Sign Up

cosmos.mx
robots.txt

Robots Exclusion Standard data for cosmos.mx

Resource Scan

Scan Details

Site Domain cosmos.mx
Base Domain cosmos.mx
Scan Status Ok
Last Scan2025-09-19T18:52:50+00:00
Next Scan 2025-10-19T18:52:50+00:00

Last Scan

Scanned2025-09-19T18:52:50+00:00
URL https://cosmos.mx/robots.txt
Redirect https://www.cosmos.com.mx/robots.txt
Redirect Domain www.cosmos.com.mx
Redirect Base cosmos.com.mx
Domain IPs 205.251.156.34
Redirect IPs 104.26.14.201, 104.26.15.201, 172.67.74.66, 2606:4700:20::681a:ec9, 2606:4700:20::681a:fc9, 2606:4700:20::ac43:4a42
Response IP 104.26.14.201
Found Yes
Hash 21b262871487b87ed71939a7281e8694e4675c085ac2b05e6c33a71f03799dc5
SimHash ca3e841082b2

Groups

sistrix

Rule Path
Disallow /

discobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

acoon

Rule Path
Disallow /

linguee

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

ip-web-crawler.com

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6
Back to top

Other Records

Field Value
sitemap https://www.cosmos.com.mx/sitemap.xml
Back to top

Warnings

  • 4 invalid lines.
Back to top