citybase.my
robots.txt

Robots Exclusion Standard data for citybase.my

Resource Scan

Scan Details

Site Domain citybase.my
Base Domain citybase.my
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-05T15:20:47+00:00
Next Scan 2025-01-03T15:20:47+00:00

Last Successful Scan

Scanned2021-08-07T14:35:59+00:00
URL http://citybase.my/robots.txt
Found Yes
Hash 2b6e5016fb209f70dcb8c70d18e5f71e92acb497228d005aeeca3aa11f355e6f
SimHash 801cd000f371

Groups

*

Rule Path
Disallow /js/

ahrefsbot

Rule Path
Disallow /

wbsearchbot/1.1

Rule Path
Disallow /

swebot/1.0

Rule Path
Disallow /

swebot/1.0; +http://swebot-crawler.net

Rule Path
Disallow /

swebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Warnings

  • 2 invalid lines.