crisis.in.ua
robots.txt

Robots Exclusion Standard data for crisis.in.ua

Resource Scan

Scan Details

Site Domain crisis.in.ua
Base Domain crisis.in.ua
Scan Status Ok
Last Scan2024-09-24T03:25:11+00:00
Next Scan 2024-10-01T03:25:11+00:00

Last Scan

Scanned2024-09-24T03:25:11+00:00
URL https://crisis.in.ua/robots.txt
Domain IPs 116.202.245.41
Response IP 116.202.245.41
Found Yes
Hash 94febce71d4bb4b76756e2b01aace9eca97944a5e25d3c92d995a6960e5fb8ea
SimHash 2b661a83a423

Groups

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

wordpress

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

*

Rule Path
Disallow /external/

*

Rule Path
Disallow /external/

Other Records

Field Value
sitemap http://crisis.in.ua/sitemap.xml
sitemap http://crisis.in.ua/sitemap.xml.gz

Comments

  • start (J)ExR
  • end (J)ExR

Warnings

  • 9 invalid lines.