citybase.mx
robots.txt

Robots Exclusion Standard data for citybase.mx

Resource Scan

Scan Details

Site Domain citybase.mx
Base Domain citybase.mx
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-09T07:09:41+00:00
Next Scan 2024-12-08T07:09:41+00:00

Last Successful Scan

Scanned2022-07-21T16:32:45+00:00
URL http://citybase.mx/robots.txt
Response IP 144.217.169.60
Found Yes
Hash 2b6e5016fb209f70dcb8c70d18e5f71e92acb497228d005aeeca3aa11f355e6f
SimHash 801cd000f371

Groups

*

Rule Path
Disallow /js/

ahrefsbot

Rule Path
Disallow /

wbsearchbot/1.1

Rule Path
Disallow /

swebot/1.0

Rule Path
Disallow /

swebot/1.0; +http://swebot-crawler.net

Rule Path
Disallow /

swebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Warnings

  • 2 invalid lines.