citybase.com.pk
robots.txt

Robots Exclusion Standard data for citybase.com.pk

Resource Scan

Scan Details

Site Domain citybase.com.pk
Base Domain citybase.com.pk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-10-05T17:14:38+00:00
Next Scan 2025-01-03T17:14:38+00:00

Last Successful Scan

Scanned2021-09-18T14:39:40+00:00
URL http://citybase.com.pk/robots.txt
Found Yes
Hash 2b6e5016fb209f70dcb8c70d18e5f71e92acb497228d005aeeca3aa11f355e6f
SimHash 801cd000f371

Groups

*

Rule Path
Disallow /js/

ahrefsbot

Rule Path
Disallow /

wbsearchbot/1.1

Rule Path
Disallow /

swebot/1.0

Rule Path
Disallow /

swebot/1.0; +http://swebot-crawler.net

Rule Path
Disallow /

swebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Warnings

  • 2 invalid lines.